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PREFACE 


This  is  the  first  of  a series  of  cooperative  Interactions  in  which  the 
Naval  Training  Equipment  Center  and  the  Air  Force  Human  Resources  Laboratory 
can  share  their  resources  for  research  on  the  development  and  use  of  simula- 
tion devices  and  training  systems.  In  this  study,  the  unique  capabilities  of 
the  Advanced  Simulator  for  Pilot  Training  (ASPT)  at  Williams  Air  Force  Base 
were  used  to  address  a problem  of  interest  to  both  the  Air  Force  and  the  Navy, 
namely  the  means  of  extending  simulation-based  training  to  the  diff icul t-to- 
fly  tasks  for  which,  to  date,  devices  equipped  with  computer  image  generation 
systems  have  not  been  used.  Several  of  the  capabilities  of  the  ASPT  were  used 
in  ways  that  make  this  study  unique.  For  instance,  this  is  the  first  study 
concerning  the  problems  of  formation  flight  performed  on  the  ASPT,  or  for  that 
matter  on  any  device  equipped  with  a computer  generated  visual  scene;  it  is 
probably  the  first  use  of  instructor-pilot  observers  In  a repeater  cockpit; 
and  this  is  the  first  time  that  it  has  been  possible  to  collect  frequency- 
domain  summary  Information  in  real-time  using  the  ASPT. 

The  efforts  of  many  people  made  the  study  possible.  Captain  Dan  F. 
Cataneo,  Airman  Randy  G.  Cline,  and  Thomas  R.  Farnan,  all  of  the  Air  Force 
Human  Resources  Laboratory,  and  Robert  A.  Greenland  of  Singer-Link  served  as 
simulator  operators  at  the  control  console;  Major  G.  Myers,  Captain  R.  C. 
Brenneman,  Captain  J.  G.  Dunbar,  and  Captain  B.  D.  Ott  of  the  97th  Plying 
Training  Squadron  served  as  subjects  In  the  first  experiment;  and  Major  S.  P. 
Hannan,  Jr.,  Major  J.  G.  Paulsen,  Jr.,  Captain  E.  B.  W.  Chun,  and  Captain 
J.  W.  Penland  of  the  Air  Force  Human  Resources  Laboratory  were  the  subjects  in 
the  second  one. 

For  success,  cooperative  research  requires  attention  to  a great  number  of 
details,  both  before  and  after  the  data  are  collected.  Captain  William  C. 
Mercer,  the  Chief  of  Naval  Education  and  Training  Liaison  Officer  at  Williams 
Air  Force  Base  energetically  provided  such  attention  and  was  an  Invaluable 
help,  for  which  we  thank  him. 

Also,  a number  of  people  helped  by  proofreading  the  report.  They  are 
Walter  S.  Chambers,  Stanley  C.  Collyer,  Paul  E.  Van  Hemel , Elizabeth  C.  W. 
Ricard,  and  Dennis  C.  Wightman. 
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SECTION  I 
INTRODUCTION 

A flight  simulator  is  a complicated  set  of  systems  that  can  sense  pilot 
control  inputs,  calculate  a given  aircraft1'?  responses  to  them,  and  then  pre- 
sent this  information  to  the  pilot.  During  real  flight,  Information  about 
the  state  of  the  aircraft  comes  from  several  sources  so  that,  to  a cockpit 
with  its  instruments  and  controls,  today's  simulators  have  attached  a variety 
of  cue-providing  subsystems  that  form  an  Interface  for  the  human  controller 
of  the  device.  Their  task  is  to  provide  visual,  auditory,  or  proprioceptive 
signals  which  pilots  can  learn  to  use  to  control  the  flight  of  their  aircraft. 
During  the  past  20  years,  we  have  seen  a dramatic  increase  of  the  number  and 
complexity  of  these  attachments  paralleling  the  development  of  more  sophisti- 
cated weapons  systems  that  are  used  with  more  complicated  tactics.  Visual 
display  systems,  for  example,  have  evolved  from  prerecorded  films  that  allowed 
little  freedom  for  simulated  flight,  to  model  board  representations  of  the 
visual  world  that  are  presented  to  the  pilot  via  a movable  television  camera, 
to  the  present  computer  image  generation  (CIG)  visual  display  systems  that 
allow  limitless  maneuvering  by  presenting  visual  scenes  that  are  stored  and 
changed  by  digital  computer.  The  techniques  for  providing  other  cues  for  air- 
craft motion  and  of  simulating  an  assortment  of  sensors  have  undergone  a 
similar  progression  of  development,  and  even  the  training  systems  that  use 
flight  simulators  are  becoming  automated. 

Modern  flight  simulators  use  digital  controllers,  and  each  of  the  develop- 
ments cited  above  has  placed  a burden  on  the  computational  capacity  of  a 
device's  computer.  For  the  most  part,  digital  computers  perform  calculations 
serially,  with  additional  calculations  requiring  additional  time.  The  CIG 
display  processing  adds  perhaps  the  greatest  number  of  calculations  to  those 
needed  to  operate  the  basic  flight  simulator.  When  CIG  visual  systems  were 
used  on  Navy  Device  2F90  and  the  Air  Force's  Advanced  Simulator  for  Pilot 
Training  (ASPT),  problems  of  controllability  caused  by  the  addition  of  this 
type  of  display  became  apparent  (O'Conner,  Shinn,  and  Bunker,  1973;  Larson 
and  Terry,  1975).  These  problems  of  device  control  were  usually  seen  during 
the  simulation  of  flying  situations  where  the  aircraft  had  to  be  flown  within 
quite  narrow  tolerances,  such  as  on  the  last  leg  of  a carrier  approach  or 
during  formation  flight. 

A temporal  gap  between  input  and  output  is  expected  in  systems  that  use 
digital  processors  as  controllers.  The  simulator's  computer  must  sample  the 
Inputs  to  the  system  and  then  calculate  its  responses,  and  in  the  case  of  a 
CIG  visual  system,  a second  computer  usually  controls  the  displaying  of  those 
responses.  For  a flight  simulator  equipped  with  a CIG  display,  this  delay  of 
visual  feedback  can  be  broken  into  two  parts  — that  related  to  the  sampling 
rate  of  the  flight  dynamics  processor  and  that  due  to  the  processing  time  of 
the  visual  display  system.  Current  flight  simulators  sample  pilot  activity 
at  15  to  30  Hz  so  that  the  time  interval  from  the  calculation  of  one  set  of 
positions  for  the  simulated  aircraft  until  the  next  is  rarely  greater  than 
66.6  milliseconds,  and  the  calculations  needed  to  create  each  true-perspective 
Image  take  about  100  milliseconds.  The  actual  total  transport  delay  may  be 
less  than  the  sum  of  these  two  values,  depending  upon  when  within  the  computa- 
tion cycle  the  values  for  the  aircraft's  new  position  are  passed  to  the  CIG 
system.  In  the  ASPT,  these  two  computers  operate  at  different  sampling  rates. 
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with  the  result  that  the  total  transport  delay  for  its  visual  scene  varies 
between  126  to  193  milliseconds  (Larson  and  Terry,  1975).  Other  cueing 
devices  of  a flight  simulator,  such  as  model  board  visual  systems,  motion 
platforms,  or  g-seats,  produce  lags  due  to  the  inertias  of  their  mechanical 
components  and  can  produce  delays  a good  deal  longer  than  those  of  the 
computer-generated  visual  displays. 

One  of  the  most  difficult  tasks  to  simulate  in  a device  equipped  with  a 
CIG  visual  system  Is  formation  flight.  Unlike  most  flying  tasks,  formation 
flight  closely  approximates  pursuit  tracking  in  that  an  image  of  a lead  air- 
craft Is  displayed  to  a pilot,  and  he  has  knowledge  of  where  he  should  be  in 
relation  to  that  craft.  One  might  then  expect  that,  with  its  visual  refer- 
ence, formation  flight  would  be  easier  than  ground-controlled  approaches  or 
instrument  landings  that  more  closely  approximate  compensatory  tracking;  but 
pilots  do  make  rather  high  frequency  control  movements  --  up  to  3 Hz  (Cyrus, 
1976)  — during  formation  flight  and  we  feel  it  is  this  tendency,  along  with 
the  phase  lag  produced  by  delays  in  the  system,  that  makes  flying  formation  so 
difficult  in  simulators  with  delays.  The  phase  lag  produced  by  a transport 
delay  is  linear  with  frequency,  causing  the  high  frequency  components  of  con- 
trol inputs  to  be  more  out  of  phase  than  the  low  frequency  ones.  When  facing 
such  a situation,  the  tendency  of  human  controllers  is  to  force  the  system 
to  a lower  gain-crossover  frequency  by  trading  response  frequency  for  ampli- 
tude. In  a sense.  It  is  the  pilot's  willingness  to  tolerate  high-frequency 
error  that  enables  him  to  keep  the  system  stable,  yet  the  requirements  of 
flying  formation  are  such  that  displayed  errors  must  be  reacted  to  quickly 
and  kept  small.  From  descriptions  of  the  operation  of  flight  simulators  with 
CIG  visual  display  systems,  it  appears  that  flying  tasks  with  those  require- 
ments of  accurate  control  of  the  aircraft  and  quick  response  to  error  are  most 
affected  by  display  delays.  This  Is  supported  by  discussions  of  the  experi- 
mental literature  of  manual  control  of  systems  with  delays  (Muckier  and 
Obermayer,  1964;  Poulton,  1974;  and  Ricard  and  Puig,  1977). 

Past  attempts  to  compensate  for  dead-time  delays  in  simulation  systems 
have,  for  the  most  part,  been  concerned  with  reducing  the  response  lags  of 
motion  platforms.  At  the  National  Aeronautics  and  Space  Administration 
Langley  Research  Center,  Parrish,  Dieudonne,  Martin,  and  Copeland  (1973)  used 
a linear  projection  of  a simulated  aircraft's  rotational  axes  positions  to 
adjust  the  signals  sent  to  the  motion  platform  of  their  device.  Later  a fil- 
ter with  a "notch"  centered  at  32  Hz  was  added  to  remove  vibration  caused  by 
the  32  Hz  iteration  rate  of  the  device's  processor.  Motion  platforms  respond 
slowly  compared  to  the  ability  of  a computer-driven  visual  display,  and  when 
the  same  technique  for  compensating  for  time  delays  is  applied  to  the  software 
that  creates  the  signals  for  a CIG  visual  system,  the  displayed  image  can  con- 
tain an  annoying  "jitter"  or  "flutter."  This  happens  because  the  linear  pro- 
jection scheme  amplifies  the  high  frequency  components  of  the  signals  sent  to 
the  display.  More  accurate  prediction  would  produce  less  jitter,  but  some 
would  occur  whenever  pilots  make  high  frequency  control  Inputs.  One  would 
think  that  software  schemes  that  provide  a phase-advance  for  the  pilot  would 
help  his  control  of  systems  with  delays,  and  Indeed  some  work  at  the  Naval 
Training  Equipment  Center  has  Indicated  that  following  a linear  projection 
scheme  with  a low-pass  filter  (which  allows  a low  frequency  phase-advance 
vrfiile  attenuating  high  frequency  jitter)  would  help  piloting  control  and 
would  potentiate  the  acquisition  of  control  skill  (Ricard,  Norman,  and 
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Collyer,  1976).  The  main  purpose  of  this  study  Is  to  test  further  the  use- 
fulness of  the  predlct-then-fil ter  notion  by  observing  trained  pilots  asked 
to  fly  formation  in  the  ASPT. 

Figure  1 should  provide  a context  for  our  changes  to  the  computer  soft- 
ware. Each  of  the  cue-providing  subsystems  of  a simulator  has  as  its  input 
the  updated  position  of  the  aircraft  as  calculated  by  the  aerodynamic  model. 
Each  subsystem  then  provides  its  own  set  of  cues  to  the  pilot  with  their 
fidelity  determined  by  that  subsystem's  capability.  When,  for  a given  task, 
control  of  the  device  must  be  made  easier,  several  options  are  available.  One 
is  modifying  the  aerodynamic  model  to  make  the  device  more  "flyable,"  and 
another  is  tinkering  (in  one  manner  or  another)  with  the  particular  subsystem 
that  seems  to  be  causing  the  difficulty.  This  latter  approach  is  the  one  we 
have  taken.  In  the  ASPT,  aircraft  parameters  are  adjusted  for  calculation 
times  in  the  flight  dynamics  software;  and  then  a filter  is  inserted  before 
the  calculation  of  the  simulated  visual  scene  to  allow  those  predicted  values 
to  be  modified  without  affecting  ones  for  the  other  cue-providing  systems. 

Our  basic  experiment  then  is  to  measure  flying  performance  while  setting  the 
time  constant  of  this  filter  to  different  values  so  that  differences  of 
piloting  control  can  be  related  to  the  spectral  content  of  the  signals  sent 
to  the  CIG  display  system. 

A second  goal  of  the  study  was  to  assess  the  usefulness  of  the  proprio- 
ceptive cueing  provided  by  the  g-seat  and  motion  platform  of  the  ASPT.  Few 
data  exist  concerning  the  advantages  offered  by  these  systems  when  the  visual 
scene  is  a computer-generated  one  and  the  flying  task  is  difficult.  This  is 
partly  because  CIG  displays  are  a new  technology  that  has  not  been  used  long, 
and  partly  because  attempts  to  use  CIG  displays  for  tasks  such  as  formation 
flight  have  not  been  particularly  successful,  usually  because  the  delay  com- 
pensation made  control  difficult  or  visual  flutter  annoying.  Using  pursuit 
tracking  tasks  very  similar  to  formation  flight.  Miller  and  Riley  (1976,  1977) 
have  shown  some  benefit  gained  by  adding  the  cues  provided  by  a simulator's 
motion  base,  at  least  for  long  delays  of  the  display.  Generally  they  found 
that  the  more  difficult  a simulation  was  to  fly,  the  shorter  was  the  delay 
that  could  be  tolerated,  and  that  activation  of  their  device's  motion  platform 
extended  the  delay  that  was  tolerable  for  the  simulation  of  a given  aircraft. 
No  data  exist  on  the  effect  of  adding  a g-seat  to  a simulation  of  flight,  but 
we  felt  that  if  its  cues  were  timely,  then  they  might  be  shown  to  be  helpful. 
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x,  y,  Z-AXIS  POSITIONS 
PITCH,  ROLL,  1 yah  ANGLES 


Figure  1.  The  cue-providing  subsystems  of  a modern  flight  simulator 
(Arrows  Indicate  the  direction  of  the  flow  of  Information). 
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SECTION  II 
EXPERIMENT  I 


METHOD 

SUBJECTS.  The  four  volunteers  who  served  as  subjects  in  this  experiment  were 
Air  Force  instructor  pilots  — members  of  the  97th  Flying  Training  Squadron 
at  Williams  Air  Force  Base,  Arizona.  All  were  instructors  for  advanced  jet 
training  in  the  T-38  aircraft  and  had  accumulated  between  1200  and  4200 
individual  hours  of  flying  time  with  a group  average  of  over  2400  hours. 

EQUIPMENT.  This  study  used  the  ASPT  --  a flying  training  research  device 
located  at  Williams  Air  Force  Base.  It  consists  of  two  simulator  cockpits 
with  Systems  Engineering  Laboratories  Model  86  computers  driving  the  motion 
bases,  g-seats,  and  computer-driven  visual  displays  for  each.  An  advanced 
instructor/operator  console  was  used  for  controlling  experiments  via  key- 
board-entered codes  and  for  monitoring  the  progress  of  individual  trials. 

The  simulator  cockpits  were  configured  to  represent  the  T-37B  aircraft  --  a 
two- seat  jet  trainer.  They  were  mounted  on  six-post,  six-degrees-of-freedom 
synergistic  motion  platforms  that  are  capable  of  60-inch  strokes  which  can 
provide  cues  of  at  least  0.6g  for  the  aircraft's  vertical,  lateral,  and 

longitudinal  accelerations  and  of  50°/second2  for  pitch,  roll,  and  yaw 
angular  accelerations.  The  cues  of  steady-state  acceleration  also  were 
simulated  by  appropriately  inflating  the  31  cells  of  the  pilot's  g-seat  and 
by  varying  the  tension  on  his  seatbelt. 

The  simulator's  CIG  system  presented  external  visual  cues  with  36-inch 
cathode  ray  tubes  and  display  optics  equipped  with  mirrors,  polarizers,  and 
beam  splitters  to  provide  i nf ini ty- focused  images.  Each  cockpit  was  partially 
surrounded  by  a mosaic  of  seven  of  these  pentagon- shaped  display  channels 

that  provided  a 300°  horizontal,  150°  vertical  field-of-view  that  essentially 
duplicated  the  field-of-view  of  the  T-37  aircraft.  After  transmission  losses 
through  the  pancake  window  optical  system,  this  display  system  could  present 
the  computer- genera ted  visual  scenes  with  highlight  brightnesses  of  up  to 
six  foot  lamberts.  The  entire  visual  display  system  is  capable  of  displaying 
2500  edges  and  of  allocating  them  over  the  14  channels  of  the  two  cockpits. 

In  this  study,  Identical  visual  scenes  were  presented  to  each  cockpit  so  that 
each  seven-tube  mosaic  was  assigned  1,000  edges  for  its  representation  of  a 
T-37  aircraft  in  the  lead  position  of  formation  flight.  A capacity  of  500 
edges  was  retained  as  a buffer  to  be  used  when  the  system  was  overloaded,  as 
occasionally,  when  parts  of  the  lead  aircraft  crossed  from  one  channel  to 
another,  the  edge-handling  limits  of  the  system  would  be  exceeded  and  parts 
of  the  CIG  picture  were  momentarily  absent  from  the  display.  Priorities  were 
assigned  so  that  the  nondisplayed  elements  were  usually  fields  placed  near  the 
horizon  to  supply  ground  texture,  but  occasionally  an  element  of  the  lead 
aircraft  would  drop  from  the  scene.  When  the  two  aircraft  were  in  their  cor- 
rect relative  positions,  these  conflicts  did  not  arise. 

The  advanced  instructor/operator  console  used  to  Interface  the  experi- 
menters to  the  simulator's  computer  system  contained  a keyboard,  two  seven- 
color  and  two  black  and  white  cathode  ray  tubes  (all  under  program  control). 
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three  closed-circuit  television  channels,  a control  stick,  plus  a number  of 
switches  and  dials.  Codes  for  the  conditions  of  the  experiment  and  commands 
to  continue  or  abort  a trial  were  entered  with  the  keyboard,  and  the  results 
were  verified  over  the  color  alphanumeric  displays.  The  achromatic  displays 
were  used  to  present  Information  concerning  the  state  of  the  two  aircraft, 
including  a graphics  representation  of  their  relative  positions.  One  of  the 
closed-circuit  television  channels  was  used  to  view  the  pilot  in  the  cockpit, 
and  the  other  two  were  used  to  display  selectable  channels  of  his  computer- 
generated visual  scene. 

TASK.  The  flying  task  used  for  this  study  was  to  maintain  the  "fingertip" 
position  off  the  right  wing  of  a lead  aircraft  (a  T-37).  This  is  a visually 

quided  task  that  requires  the  pilot  to  hold  a position  30°  back  from  the 
lead  aircraft  while  maintaining  a three-foot  wing  tip  clearance.  When  this 
position  is  obtained,  the  lead  aircraft  appears  as  in  Figure  2 (copied  from 
Air  Training  Manual  51-4)  which  is  a fair  representation  of  the  high  fidelity 
image  of  the  lead  aircraft  used  by  the  ASPT.  Several  of  the  visual  references 
that  pilots  are  trained  to  use  to  hold  the  fingertip  position  have  been  indi- 
cated In  the  figure:  (1)  the  lead  pilot's  head  is  kept  aligned  with  the  out- 
side flap  hinge  and  the  radio  antenna  of  the  lead  aircraft,  (2)  the  top  1/3 
and  bottom  2/3  of  the  lead  aircraft's  right  wing  are  visible,  (3)  the  trailing 
edge  of  the  lead  aircraft's  opposite  elevator  is  barely  visible,  as  is  (4)  the 
top  part  of  an  inverted  triangular  ejection  warning  sign  located  on  the  fuse- 
lage of  the  lead  aircraft  just  forward  of  the  pilot.  For  straight-and-level 
flight,  the  fingertip  position  can  be  maintained  with  a constant  throttle 
setting  and  small  movements  of  the  control  stick,  but  during  maneuvers  like 
turns,  throttle  changes  must  be  made  in  order  to  maintain  the  position. 


Figure  2.  Pilots'  view  of  the  T-37  aircraft  In  the  lead  position  when 
flying  the  finger-tip  position  off  Its  right  wing  (Taken  from 
ATCM  51-4). 
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EXPERIMENTAL  DESIGN.  Our  Intention  was  to  examine  the  effect  of  selectively 
filtering  the  drive  signals  sent  to  the  CIG  system  processor  and  the  effects 
such  filtering  might  have  on  a pilot's  use  of  the  nonvisual  cues  of  aircraft 
motion  available  In  modern  flight  simulators.  To  that  end,  this  data  collec- 
tion was  designed  as  a three- factor,  mixed -effects  factorial  experiment  where 
differences  due  to  subjects  were  considered  a random  effect  and  the  effects 
produced  by  the  types  of  motion  cueing  and  settings  of  the  display  signal 
filters  were  considered  fixed.  Three  conditions  of  motion  cueing  were 
selected  --  corresponding  to  the  presentation  of  just  the  visual  scene,  or 
to  the  addition  of  the  proprioceptive  cues  provided  by  either  the  g-seat  or 
the  motion  platform  of  the  ASPT.  Six  settings  for  the  signal  filters  were 
included;  these  represented  half-power  points  for  the  low  pass  filters  of  1/4, 
1/2,  3/4,  1,  and  2 Hz,  and  an  unfiltered  condition.  Thus  was  formed  a three- 
by-six-factorial  whose  18  combinations  of  motion  and  filter  conditions  were 
block  randomized  with  a different  order  of  presentation  for  each  subject. 

Each  subject  completed  three  of  these  18-trial  blocks  during  the  experiment. 

Despite  the  fact  that  our  subjects  were  expert  instructor  pilots,  they 
experienced  considerable  difficulty  in  adjusting  to  the  characteristics  of 
the  simulator.  This  was  partly  because  the  presence  of  the  transport  delay 
made  the  formation  flying  task  difficult,  partly  because  the  computer  system 
failed  several  times  during  the  first  day  or  two  of  the  experiment,  and 
Partly  because  the  pilots  were  not  accustomed  to  the  math  models  for  certain 
subsystems  of  the  device,  particularly  the  one  that  calculated  changes  of 
airspeed  in  response  to  small,  quick  changes  of  the  throttles.  For  this 
reason,  we  expected  performance  to  improve  throughout  the  experiment  and  our 
analyses  of  variance  included  a "trials"  effect  to  remove  this  variability 
from  the  error  terms. 

A secondary  objective  of  the  study  was  to  compare  ratings  obtained  from 
both  the  pilot  flying  a maneuver  and  an  observer  in  the  second  cockpit. 

While  both  raters  saw  the  visual  display  of  the  lead  aircraft,  only  the 
Pilot  had  his  g-seat  or  motion  base  activated,  so  that  comparison  of  the 
pilots'  and  observers'  ratings  would  me'.sure  the  contribution  that  nonvisual 
cues  of  motion  had  on  the  pilots'  judgments  of  the  adequacy  of  a simulated 
flight.  For  these  comparisons,  we  used  the  same  analysls-of-variance  model 
mentioned  earlier.  These  rating  data  were  collected  as  the  experiment 
progressed  and  for  the  pilots  at  least,  they  fit  the  block-randomized  design. 

EXPERIMENTAL  VARIABLES. 

Filter  Settings.  Previous  work  (Rlcard,  Norman,  and  Collyer,  1976;  and  tests 
at  the  Air  Force  Human  Resources  Laboratory)  has  established  that  an  optimal 
setting  for  the  filters  for  the  CIG  display  signals  would  be  in  the  range  1/2 
to  1 Hz,  so  for  this  test  of  the  utility  of  the  signal  filtering  n*t.1on,  a 
single-pole  Butterworth  filter  (see  Sterns,  1975  or  Oppenheim  and  Shafer, 

1975)  routine  was  developed.  To  be  consistent  with  previous  work  performed 
at  the  Naval  Training  Equipment  Center,  we  used  a first-order,  low-pass  filter 
with  the  Butterworth  configuration  selected  for  its  smooth  amplitude  and  phase 
response.  Only  three  degrees-of-freedom  of  the  simulated  aircraft  were  fil- 
tered — roll  angle,  pitch  angle,  and  z-axls  — because  the  preliminary  anal- 
ysis of  other  work  performed  at  the  Air  Force  Human  Resources  Laboratory 
(Cyrus,  1976)  indicated  that  only  the  responses  along  these  axes  contained 
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enough  high  frequency  energy  to  create  problems  of  "flutter"  in  the  visual 
display.  When  the  filter  routine  was  integrated  into  the  ASPT's  software,  it 
was  placed  after  the  visual  prediction  algorithm  (see  Larson  and  Terry,  1975), 
and  it  therefore  affected  only  the  visual  display,  not  other  aspects  of  the 
simulated  flight  dynamics.  All  translational  and  rotational  variables  were 
time-adjusted  for  the  transport  delay  In  the  ASPT  CIG  visual  sy:tem,  then 
these  mentioned  were  filtered,  and  finally  all  were  processed  by  the  CIG 
system.  Specifically,  each  variable  was  predicted  according  to  tlie  following: 

xn+1  = xn  + h/2(3*n+l  - *n)  where  x is  the  variab1e  bein9  adjusted,  * its 

derivative,  the  subscript  refers  to  a value  at  a point  in  time,  and  h is  the 
integration  interval.  To  acconmodate  different  Iteration  rates  for  the  pro- 
cessing of  the  flight  dynamics  and  the  visual  image,  the  prediction  span  h is 
variable  in  the  ASPT.  We  deemed  It  premature  to  try  to  select  individual 
settings  for  the  filter  for  each  axis,  so  for  this  test,  the  three  aircraft 
parameters  that  were  adjusted  were  passed  through  identical  filters.  The 
method  developed  to  accomplish  the  integration  and  filtering  for  these  axes 
has  been  presented  by  Cyrus  (1977)  as  a general  technique  for  compensating 
CIG  system  transport  delays.  Its  advantages  are  that  the  calculations  it 
requires  can  be  performed  simply  and  quickly,  and  It  would  be  easy  to  imple- 
ment either  in  hardware  or  software.  The  technique  does  require  that  con- 
stants be  set  that  may  be  a function  of  the  type  of  flying  task  being  simu- 
lated, hence  the  need  for  studies  like  this  one. 

In  evolving  this  approach  to  compensation  for  transport  delays  in  visual 
display  systems,  we  assumed  that  the  Information  used  by  the  pilot  for  con- 
trolling the  aircraft  and  the  noise  generated  by  software  prediction  schemes 
are  different  functions  of  frequency.  More  specifically,  we  assumed  that  both 
are  monotonic  functions  somewhat  similar  to  those  depicted  in  Figure  3,  in 
that  most  of  the  useful  information  in  the  signal  is  low  frequency  and  that 
the  more  annoying  noise  is  high  frequency.  The  manipulation  of  the  break 
frequencies  of  the  signal  filters,  then,  is  an  empirical  attempt  to  find  the 
signal-to-noise  crossover  point  of  Figure  3.  By  setting  the  break  frequency 
of  the  filters  there  and  removing  the  high  frequency  components  from  the 
signals  sent  to  the  CIG  display  processor,  we  should  be  able  to  improve  the 
"quality"  of  the  visual  information  presented  to  the  pilot,  and  presumably, 
his  flying  performance  also. 

Motion  Cues.  Because  the  two  aircraft  were  In  their  correct  positions  at  the 
start  of1  a trial  and  because  we  used  Instructor  pilots  for  the  formation 
flying  task,  we  expected  that  our  subjects  would  have  to  produce  only  small 
changes  of  piloting  control  and  aircraft  state  parameters  in  order  to  control 
the  simulator,  and  that  such  a situation  would  enhance  the  usefulness  of  those 
subsystems  of  flight  simulators  that  provide  nonvisual  cues  of  aircraft 
motion.  If  a trial  was  flown  well  and  the  simulated  aircraft  was  kept  in  its 
proper  fingertip  position,  the  expected  motions  of  the  cockpit  would  be  small 
and  the  relative  scaling  of  the  signals  sent  to  the  actuators,  while  normally 
less  than  unity  so  that  effective  cueing  can  be  provided  for  large  g-forces, 
could  for  this  test  be  set  one-for-one.  To  that  end,  the  mathematical  model 
for  the  signals  sent  to  the  motion  platform  was  changed.  The  "gravity  align" 
subroutine  — which  keeps  the  resultant  simulated  external  force  vector 
aligned  with  the  normal  earth  vector  — was  kept  active,  but  no  additional 
cues  for  the  translational  motions  of  the  aircraft  were  provided.  Together 
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these  changes  of  the  software  for  the  motion  platform  were  designed  to  opti- 
mize the  simulation  of  small  rotational  responses.  That  they  were  successful 
was  indicated  by  the  ability  of  instructor  pilots  assigned  to  the  Air  Force 
Human  Resources  Laboratory  to  identify  correctly  the  direction  of  small 

( < 1 0° ) platform  inclinations  and  by  their  agreement  that  indeed  the  motion 
platform  was  perceived  to  move  before  the  CIG  visual  display  responded  when 
small  control  inputs  were  entered  from  the  cockpit.  Details  about  the  ASPT 
motion  system  are  provided  by  Kron  (1975a). 

To  increase  the  ability  of  the  g-seat  to  act  as  a signaling  device,  its 
software  was  changed  to  provide  a nonlinear  signal  for  the  position-following 
bellows  system.  This  was  accomplished  by  creating  as  the  forcing  function 
for  the  bellows,  a weighted  average  of  aircraft  velocity  and  acceleration 
values.  The  amount  of  lead  which  the  g-seat  could  be  provided  was  thus  con- 
trollable, and  for  this  study  was  subjectively  set.  As  in  the  case  of  the 
adjustments  to  the  action  of  the  simulator's  motion  system,  the  cues  of 
rotational  motion  were  considered  to  be  more  important  than  those  of  transla- 
tional motion.  Kron  (1975b)  has  also  described  the  g-seat  of  the  ASPT. 


Figure  3.  Hypothesized  relation  of  Information  useful 

for  flight  control  and  prediction  scheme  noise 
In  the  computer-generated  Image  as  a function 
of  signal  frequency. 
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PERFORMANCE  MEASUREMENT.  We  collected  two  types  of  data:  objective  measures 
of  the  pilot's  performance  of  the  flying  maneuver,  and  subjective  impressions 
both  of  his  performance  and  of  the  jitter  of  the  CIG  display.  The  latter 
measures  were  ratings  collected  from  both  the  pilot  and  an  observer  after 
each  trial.  For  the  formation  flying  task,  the  objective  data  included 
measures  of  the  translational  motions  of  the  aircraft  (x-,  y-,  and  z-axis 
errors  measured  as  differences  of  position  of  the  two  aircraft),  measures 
of  its  rotational  motion  (pitch,  roll,  and  yaw  angles,  again  measured  as 
differences),  and  measures  of  the  position  of  the  aileron  and  elevator  co,  - 
trol  stick,  and  the  positions  of  the  rudder  pedals  and  of  the  left  and  right 
throttles.  The  measures  of  the  lag  aircraft's  absolute  position  in  space 
were  obtained  relative  to  the  position  of  the  lead  aircraft;  to  adjust  them 
to  be  errors  from  the  Ideal  position  for  the  lag  craft,  the  following  con- 
stants should  be  added:  x-axis  = -21.25  feet,  y-axis  = 36.78  feet,  and  z-axis 
= 3.00  feet.  After  the  first  20  seconds  of  a trial,  these  eleven  parameters 
were  sampled  15  times  per  second  and  recorded  for  137  seconds.  The  perform- 
ance measurement  routin'  developed  for  this  study  obtained  both  time-history 
and  frequency-domain  summary  information  for  these  eleven  variables.  These 
summaries  represent  linear  transformations  of  the  information  contained  in 
the  time  histories,  and  for  each  parameter  sampled,  we  calculated  its  mean 
and  variance.  Each  variable  was  also  passed  through  five  single-pole 
Butterworth  filters  with  half-power  points  of  1/8,  1/4,  1/2,  1,  and  2 Hz  to 
obtain  a crude  estimate  of  its  power  spectral  density  function.  At  the  end 
of  the  day's  testing,  all  of  these  records  were  stored  on  magnetic  tape  along 
with  the  information  needed  to  Identify  each  pilot,  trial,  and  factor  combi- 
nation. 

Ratings  provided  the  subjective  assessments  of  the  flights  and  of  the  CIG 
system's  visual  scene.  At  the  end  of  each  tri-i,  both  the  pilot  and  the  ob- 
server were  asked  to  rate  the  flignt  usi.ig  the  scales  presented  in  Tables  A-l, 
A-2,  and  A-3  of  Appendix  A.  One  rating,  using  a 12-point  numerical  scale, 
was  used  to  indicate  how  well  the  lag  aircraft  appeared  to  maintain  the 
fingertip  position,  and  another  rating,  a nine-point  letter  scale,  was  used 
to  appraise  the  "usefulness"  of  the  CIG  visual  scene.  The  information  needed 
to  make  the  numeric  ratings  was  presented  on  each  instructor  pilot's  score 
sheet  (Table  A-4  of  Appendix  A)  so  that,  as  an  example,  a flight  rated  "good" 
would  receive  scores  of  7,  8,  or  9 which  would  represent  a flight  where  the 
lag  aircraft  was  held  approximately  not  more  than  2 feet  high,  4 feet  low, 

3 feet  forward  or  backward,  1 foot  to  the  left,  and  5 feet  to  the  right  of 
the  ideal  position.  These  criteria  were  developed  through  discussions  with 
Instructor  pilots  at  Williams  Air  Force  Base  and  after  considering  the 
geometry  of  flying  formation  — l.e.,  moving  the  lag  upward  and  to  the  left, 
for  example,  would  be  much  more  hazardous  than  moving  it  downward  and  to  the 
right.  The  other  ratings,  recorded  as  letters  to  avoid  confusion  with  the 
numeric  scale,  were  a bit  more  loosely  defined.  They  were  included  to  assess 
the  annoyingness  of  the  "flutter"  In  the  CIG  system's  visual  display.  Init- 
ially the  instructor  pilots  were  ignorant  of  the  purpose  of  varying  the 
settings  of  the  display  signal  filters,  but  after  the  first  18  trials,  they 
were  informed  that  it  was  the  visual  flutter  that  we  were  trying  to  remove 
from  the  display  and  they  were  requested  to  direct  their  ratings  to  that 
aspect  of  the  visual  scene. 
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PROCEDURE.  Several  events  took  place  before  the  experiment  proper  began.  One 
of  these  was  to  make  fllghtpaths  for  the  lead  aircraft  with  which  the  Instruc- 
tor pilots  could  fly  formation.  The  Instructor's  console  of  training  simula- 
tors is  often  equipped  with  a control  stick  which  can  be  used  to  change  the 
position  of  the  pilot's  aircraft  or  to  "fly"  the  image  displayed  to  him. 
Usually  there  is  no  throttle  for  this  control  and  the  aerodynamic  model  that 
generates  responses  to  inputs  from  the  instructor's  station  is  simpler  than 
the  one  that  responds  to  inputs  from  the  cockpit  of  the  simulator.  Because 
the  ASPT  is  equipped  with  two  cockpits,  each  with  its  own  CIG  visual  display 
mosaic,  we  decided  to  prerecord  flights  for  the  lead  aircraft  and  thus  be 
able  to  use  both  cockpits  for  real-time  data  collection.  An  instructor  pilot 
assigned  to  the  Human  Resources  Laboratory  flew  a number  of  flights  during 
which  he  performed  gentle  climbs,  dives,  and  turns.  These  flights  were  re- 
corded and  then  used  to  drive  the  CIG  visual  system  to  create  the  flightpath 
for  the  lead  aircraft  for  our  formation  flying  task.  During  the  data  collec- 
tion, the  prerecorded  flight  controlled  the  image  of  the  pilot's  view  of  the 
lead  craft  for  both  cockpits.  In  this  way,  the  lead  aircraft  could  be  made 
to  fly  realistically  while  freeing  the  second  cockpit  for  an  observer.  Twelve 
such  flights  were  recorded,  and  from  this  set,  six  were  chosen  for  use  during 
the  experiment.  Two  criteria  were  used  for  their  selection:  first,  that  the 
instructor  pilots  assigned  to  Air  Force  Human  Resources  Laboratory  judged 
that  the  particular  flightpath  was  not  too  difficult  for  formation  flight, 
and  second,  that  the  selected  flights  presented  fewer  synchronization  problems 
than  the  rejected  ones.  If  a trial  was  started  at  an  inopportune  time,  the 
calculation  of  the  position  for  the  lead  aircraft  was  begun  before  that  for 
the  lag  aircraft  and  the  CIG  system's  image  "jumped"  ahead.  It  was  for  this 
reason  that  a 20-second  dead-time  was  included  at  the  start  of  each  trial. 

It  gave  the  subjects  time  to  catch-up  to  the  lead  aircraft  before  data  were 
recorded.  Some  of  the  prerecorded  flights  seemed  to  magnify  this  synchroni- 
zation problem,  and  these  were  the  ones  we  deleted.  The  advantage  gained  by 
using  prerecorded  flights  was  the  improved  dynamic  response  of  the  lead  air- 
craft which  the  better  math  model  allowed.  However,  a simulator  with  two 
"active"  cockpits  has  a disadvantage.  The  effective  delay  from  control  input 
in  one  cockpit  to  display  and  then  response  from  the  other  cockpit  would  now 
be  doubled.  A 100  to  200  millisecond  visual  system  transport  delay  becomes 
a 200  to  400  millisecond  total  system  delay,  and  the  problems  of  piloting 
control  are  made  even  more  difficult.  We  might  note  that  this  was  not  the 
case  in  this  study  as  only  the  pilot's  cockpit  could  provide  inputs  to  the 
CIG  visual  system. 

Personnel  at  the  Air  Force  Human  Resources  Laboratory  also  performed  the 
tasks  of  software  integration,  i.e.,  of  developing  and  Inserting  into  the 
simulation  program  the  filtering  and  performance  measuring  routines,  and 
of  setting  the  lead  times  and  gains  for  the  g-seat  and  motion  platform  of 
the  ASPT.  They  also  developed  the  rating  scales  specifically  for  the  forma- 
tion flying  task  and  created  the  block  randomized  orders  of  experimental  con- 
ditions for  each  pilot.  This  last  task  was  performed  on  a Wang  computer  at 
the  Human  Resources  Laboratory  using  programs  developed  there. 

Prior  to  the  collection  of  data,  the  instructor  pilots  were  briefed  about 
the  nature  of  the  experiment  and  of  what  would  be  required  of  them,  and  on 
normal  operating  and  safety  procedures  for  the  ASPT.  Along  with  viewing  a 
safety  film,  they  had  an  opportunity  to  "fly"  the  ASPT  for  a while  to  see  how 
it  operated. 
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The  experiment  consisted  of  ten  four-hour  testing  sessions  with  two  in-  j 

structor  pilots  present  each  session.  Each  served  as  pilot  and  then  observer  ; 

for  about  half  of  each  session,  with  each  serving  as  observer  for  every  other 
at  least  twice  during  the  five  days  of  testing.  Initially  we  had  hoped  to  1 

complete  one  block  of  18  trials  per  day,  but  this  proved  to  be  overly  optimis-  * 

tic  so  that  only  three  such  blocks  were  finished  by  the  end  of  the  data  col- 
lection. At  the  beginning  of  a testing  session,  the  simulator  was  checked  to 
see  that  it  was  set  appropriately  for  this  experiment,  and  then,  while  the 
device  was  being  prepared,  the  pilot  and  observer  were  given  their  data  sheets 
and  asked  to  enter  the  cockpits.  Radio  contact  was  established  with  each 
pilot  and  was  the  means  of  pilot-experimenter  communication  used  during  all  of 
the  tests.  Each  trial  Included  a one-to-two  minute  period  while  experimental 
condition  codes  were  entered  at  the  Instructor/operator  console,  and  then, 
when  the  operator  judged  the  pilot  ready,  a key  press  actuated  the  trial  pro- 
gram, starting  a 157-second  trial  with  data  recorded  during  the  last  137 
seconds.  All  trials  were  started  with  the  fully-fueled  T-37  flying  at  an 
airspeed  of  190  knots  at  an  altitude  of  15,000  feet.  At  the  end  of  each,  one 
of  the  instructor  pilots  assigned  to  the  Air  Force  Human  Resources  Laboratory 
made  the  judgment  of  whether  or  not  the  flight  should  be  considered  controlled 
enough  to  produce  reasonable  oata,  and  then  the  recordings  were  either  entered 
into  the  data  file  for  this  study  or  were  deleted.  Conditions  under  which 
control  was  lost  were  repeated  immediately.  Trials  for  which  the  motion 
platform  had  to  be  activated  took  a bit  longer  than  others  as  the  walkway  to 
the  cockpit  had  to  be  retracted  and  the  motion  platform  had  to  be  raised  to 
its  starting  level.  All  of  these  events  allowed  an  average  time-per-trial  of 
five  to  ten  minutes,  a time  that  was  gradually  reduced  throughout  the  experi- 
ment. Until  It  was  time  to  switch  places,  the  pilot  and  observer  remained  in 
their  cockpits,  resting  between  trials. 

As  performance  data  were  collected,  they  entered  a disk-file  that  was 
stored  on  magnetic  tape  at  the  end  of  each  day.  After  the  experiment  was 
completed,  a listing  was  made  of  this  file,  and  the  magnetic  tape  was  taken  to 
the  Naval  Training  Equipment  Center  for  analysis. 

At  the  end  of  the  experiment,  we  discovered  that  a total  of  13  trials  had 
been  omitted  because  of  human  errors  while  entering  the  condition  codes  and 
trial  numbers  at  the  instructor  console.  The  analysis  procedures  we  used  re- 
quired a complete  set  of  data  from  each  subject,  so  that  each  subject's 
missing  information  was  estimated  using  measures  of  his  performance  under 
similar  conditions.  For  no  subject  was  one  of  the  18  different  experimental 
conditions  skipped  more  than  once,  so  that  we  always  had  two  sets  of  measures 
with  which  to  estimate  the  third. 

RESULTS  AND  DISCUSSION 

To  keep  this  discussion  simple,  the  details  of  each  analysis  of  variance 
are  presented  at  the  end  of  the  report  and  will  be  discussed  only  generally. 

Based  on  our  results,  it  may  be  desirable  to  make  similar  adjustments  to 
other  flight  simulators;  so  for  this  reason,  all  of  the  statistically 
significant  effects  related  to  the  main  variables  of  the  study  are  presented 
graphically  for  reference. 
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OBJECTIVE  MEASUREMENTS.  Tables  B-l  to  B-ll  of  Appendix  B present  the  results 
of  the  analyses  of  variance  performed  on  the  means  and  variances  of  the  11 
Parameters  we  measured.  Each  table  presents  the  analysis  of  the  average  value 
of  the  measure  and  then  the  analysis  of  its  variance.  In  these  tables,  the 
variables  we  manipulated  are  designated  by  A,  B,  and  C and  each  subject's  data 
were  considered  a "block,"  so  to  decode  the  tables:  A = conditions  of  motion 
cueing,  8 = settings  of  the  filters,  C = trials,  and  a block  effect  reflects 
differences  due  to  the  pilots.  For  an  effect  significant  at  the  p<05  level 
of  confidence  or  above,  these  tables  give  the  probability  of  the  occurrence 

o 

of  its  F-ratio  and  its  proportion  of  the  total  sum  of  squares,  n (Eta 

o 

Squared).  To  provide  some  perspective,  n for  each  residual  variance  term  is 
also  given.  Generally  speaking,  an  advantage  was  gained  by  structuring  the 
data  collection  as  a blocked  randomized  factorial  experiment  with  trials  as  a 
factor.  Often  both  the  blocks  and  trials  effects  produced  significant 
F-ratios,  and  as  we  were  able  to  remove  this  variance  from  the  residual  terms, 
the  other  tests  were  undoubtedly  made  more  sensitive.  Several  observations 
can  be  made. 

First,  the  residual  (error)  variances  of  these  analyses,  that  variance 
which  could  not  be  parsed  into  any  of  the  effects,  were  often  quite  large. 

For  the  average  values  of  the  measures,  for  example,  the  error  terms  accounted 
for  67  percent  of  the  total  sums  of  squares,  and  for  the  variances,  the  resid- 
ual variances  accounted  for  59  percent.  Typically  none  of  the  effects  we 
could  analyze  were  statistically  significant  if  the  residual  term  had  an 
2 

n greater  than  0.70.  Usually  those  analyses  of  variance  that  produced 

2 

significant  F-ratios  had  error  terms  for  which  n was  in  the  range  0.50  to 
0.65.  In  only  three  analyses  (the  ones  of  the  variances  of  the  positions  of 
both  throttles  and  of  the  mean  position  of  the  left  one)  were  we  able  to 
reduce  the  residual  term  to  less  than  50  percent  of  the  total  sum  of  squares, 
so  that  obviously  a good  deal  of  the  variability  of  our  pilots'  performances 
was  not  accounted  for  by  the  conditions  of  the  study. 

Related  to  the  large  amount  of  unaccounted  variance  is  the  presence  of  a 
trials  effect  in  the  analysis  of  many  of  the  measures.  This  was  seen  in  the 
analyses  of  the  lag  aircraft's  attitude  (relative  to  the  lead  craft)  and  in 
the  measures  of  piloting  control.  While  the  trials  effects  were  statistical- 

p 

ly  reliable,  they  were  not  large,  with  n values  of  only  0.10  to  0.12. 

Usually  this  difference  of  average  performance  was  seen  between  the  first  set 
of  18  trials  and  the  next  two  sets.  Our  instructor  pilots  were  naive  to  the 
ASPT  so  that  an  effect  due  to  sequential  testing  was  quite  expected.  It  was 
included  in  the  analyses,  as  was  the  blocks  effect,  to  reduce  the  size  of  the 
error  terms.  Usually  the  effect  due  to  progressive  trials  was  quite  relia- 
ble, with  small  probabilities  of  occurring  by  chance,  yet  some  measures,  as 
the  difference  of  translational  position  of  the  two  aircraft  did  not  show 
this  trend. 

The  lack  of  a trials  effect  in  the  analyses  of  the  measures  of  the  posi- 
tion in  space  of  the  lag  aircraft  leads  to  a third  observation:  that  those 
measures  were  relatively  insensitive  to  the  manipulations  of  this  study.  On 
only  one  test  --  that  for  the  effect  of  different  filter  settings  on  the 
variance  of  the  x-axis  difference  of  position  of  the  two  aircraft  — was  a 
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significant  result  obtained  (shown  in  Figure  4),  and  this  was  probably 
spurious.  This  was  not  one  of  the  variables  we  filtered,  and  it  also  is  one 
where  we  would  expect  little  or  no  such  effect  to  be  found.  Our  impressions 
during  the  collection  of  the  data  were  that  the  actual  flightpaths  taken  by 
the  lag  aircraft  were  a bit  too  variable  for  the  measurement  of  any  differ- 
ences of  the  pilots'  ability  to  control  the  simulator,  and  the  analyses  of 
variance  confirmed  these.  Having  the  lead  aircraft  undergo  climbs,  dives, 
and  turns  and  requiring  the  pilots  to  maintain  their  aircraft  in  a given 
position  relative  to  the  lead  brought  to  the  fore  their  unfamiliarity  with 
the  simulation  system. 


Filter  Break  Frequency  in  Hz 


Figure  4.  Variance  of  the  x-axIs  difference  of  position  of  the 
lead  and  lag  aircraft  as  a function  of  filter  setting. 

In  this  and  the  subsequent  figures,  the  - symbol  represents 
the  break  frequency  or  the  unflltered  condition,  although 
for  this  simulation,  the  Myqulst  frequency  of  the  ASPT 
CIG  system  Is  IS  Ha. 
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Along  with  the  z-axis  difference  of  position,  we  filtered  the  differences 
of  the  aircrafts'  pitch  and  roll  angles,  and  of  these,  the  variance  of  the 
pitch  angle  measure  showed  an  effect  of  this  filtering.  The  variability  of 
the  lag  aircraft's  pitch  angle  was  reduced,  and  so  was  the  variability  of  the 
pilots'  longitudinal  and  lateral  movements  of  their  control  stick.  That  our 
manipulations  of  the  display  actually  ^fected  piloting  control  is  comforting, 
given  the  adaptation  to  the  simulation  system  required  of  the  pilots.  These 
measures  presented  as  a function  of  the  break  frequency  of  the  display  signal 
filters  are  depicted  in  Figures  5 and  6.  All  of  these  functions  display  a 
reduction  of  variance  as  the  break  frequency  of  the  filter  is  increased  up  to 
about  3/4  to  1 Hz.  For  the  variance  of  the  differences  of  pitch  angle,  there 
is  a hint  that  the  variance  then' increases  under  the  no-filtering  condition, 
although  that  value  does  not  reliably  differ  from  those  produced  by  filtering 
in  the  range  of  3/4  to  2 Hz.  The  difference  is  not  statistically  significant, 
but  it  does  hold  the  possibility  that  more  sensitive  measurements  may  find 
that  the  measures  of  aircraft  attitude  display  a U-shaped  function  across 
filter  settings.  Those  for  the  variances  of  the  positions  of  the  control 
stick  definitely  appear  to  reach  an  asymptote  at  about  1 Hz  with  no  indication 
of  a high  frequency  elevation.  Our  filtering  of  the  display  signals  not  only 
encouraged  the  pilots  to  use  the  aircraft's  control  stick  differently,  but 
also  led  to  differences  of  their  use  of  the  throttles.  The  average  positions 
of  the  left  and  right  throttles  are  presented  in  Figure  7.  Although  the 
significant  differences  of  throttle  position  were  seen  in  the  average  values, 
not  in  the  variance,  of  these  positions,  these  functions  appear  similar  to 
the  previous  ones  where  variances  were  reduced  as  the  display  was  made  more 
responsive  until  a filter  setting  of  3/4  to  1 Hz  was  reduced. 

Most  of  the  significant  results  we  obtained  were  with  measures  of  the  dif- 
ferences of  aircraft  attitude  or  of  pilot  control  inputs.  The  majority  of 
these  results  were  effects  due  to  differences  between  pilots  or  blocks  of 
trials,  but  some  were  relatable  to  the  variables  of  the  study.  Of  the  22 
analyses  of  variance  performed  on  the  means  and  variances  of  our  measures, 
five  effects  were  found  to  be  significant  but  difficult  to  interpret.  One  was 
the  effect  of  filtering  on  the  variance  of  the  x-axis  difference  of  aircraft 
position  mentioned  earlier.  Another  was  the  only  effect  that  the  conditions 
of  motion  cueing  produced  — this  was  on  the  variance  of  the  differences  of 
yaw  angle.  Again  why  the  filtering  we  did  should  affect  this  measure  is  not 
clear.  In  addition  to  these  two  main  effects,  three  interactions  produced 
significant  F-ratios  that  seemed  arbitrary.  These  were  a motion  cueing  by 
trials  interaction  of  the  mean  of  the  roll  angle  measure,  a motion  cueing  by 
filter  setting  by  trials  effect  seen  in  the  variance  of  the  differences  of  yaw 
angle,  and  a motion  cueing  by  filter  setting  interaction  of  the  variance  of 
the  position  of  the  rudder  pedals.  None  of  these  effects  would  have  been  pre- 
dicted nor  are  they  particularly  Interpretable. 
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Filter  Break  Freouency  in  Hz 

Figure  5.  Variance  of  the  difference  of  pitch  angle  of  the  lead 
and  lag  aircraft  a function  of  filter  setting. 


Filter  Break  Frequency  in  Hz 

Figure  6.  Variance  of  the  pilots'  longitudinal  and  lateral-axis 

movements  of  their  control  stick  as  a function  of  filter 
setting. 
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Filter  Break  Frequency  in  Hz 


Figure  7.  Average  position  of  the  left  and  right  throttles  as  a function 
of  filter  setting. 


RATING  MEASURES.  The  subjective  ratings  of  the  flight'  provided  by  the  pilots 
and  observers  were  analyzed  in  the  same  manner  as  the  oojective  measures  and 
these  results  are  presented  in  Tables  B-12  to  B-15  of  Appendix  B.  In  all  four 
of  these  analyses,  the  blocks  and  trials  effects  were  significant,  indicating 
that  the  pilots  and  observers  not  only  differed  In  their  ratings,  but  tended 
to  give  higher  ratings  during  the  last  two  sets  of  trials.  Of  the  two  sets 
of  ratings,  those  based  on  the  flights  were  more  sensitive  to  the  variables 
we  manipulated  in  this  study  than  were  those  based  on  the  noise  of  the  visual 
image.  The  ratings  of  flight  control  given  by  the  pilots  are  depicted  in 
Figure  8,  and  those  made  by  the  observers  In  Figure  9.  In  these  ratings, 
effects  of  both  filter  settings  and  conditions  of  motion  cueing  were  found. 

For  the  ratings  of  a flight  made  by  the  pilot,  there  is  no  difference  between 
the  subjective  Impressions  obtained  from  the  visual  display  alone  and  with 
the  g-seat  added.  Both  conditions  displayed  their  highest  ratings  at  a break 
frequency  of  3/4  Hz  with  a slight  trend  toward  lower  ratings  as  the  display 
was  made  more  responsive.  The  ratings  for  the  condition  with  the  motion 
platform  activated  were  on  the  average  lower  and  displayed  less  of  an  effect 
of  changing  the  filter  setting.  For  the  ratings  of  these  flights  made  by  the 
observers,  similar  trends  are  evident.  They  judged  flights  to  be  most  con- 
trolled when  the  pilot's  g-seat  was  activated  and  least  controlled  when  the 
motion  platform  was  used.  While  the  differences  due  to  conditions  of  motion 
cueing  are  significant,  the  ones  across  display  filter  settings  were  not 
significant  in  this  set  of  ratings  provided  by  the  observers. 
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Filter  Break  Frequency  in  Hz 

Figure  8.  Pilots'  ratings  of  their  flight  control  as  a function 

of  filter  setting.  The  parameter  represents  the  different 
conditions  of  motion-cueing  (V  ■ visual  display  only, 

V+G  » visual  display  plus  g-seat,  and  V+M  - visual  display 
plus  notion  platform). 


Filter  Break  Frequency  in  Hz 

Figure  9.  Observers'  ratings  of  flight  control  as  a function  of 

filter  setting.  The  parameter  Is  the  same  as  In  Figure  8. 
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The  other  sets  of  ratings  (of  the  noisiness  of  the  CIG  display)  were  a bit 
disappointing  in  that  no  differences  related  to  the  variables  of  the  study 
were  found.  The  subjects  were  not  really  informed  about  the  nature  of  the 
problem  of  jitter  in  CIG  displays,  and  it  took  them  some  time  to  become  sensi- 
tive to  it.  This  learning  is  reflected  in  the  significant  trials  and  blocks 
effects  in  these  data.  Most  of  the  complaining  about  flutter  or  noise  in  CIG 
visual  systems  comes  from  people  quite  familiar  with  a particular  display, 
so  we  suspect  that  some  training  would  have  made  these  ratings  more  useful. 

Because  some  of  the  flying  control  data  indicated  that  an  optimal  filter 
setting  could  be  found  that  would  allow  better  flying  control  performance  than 
an  unfiltered  display  would,  and  because  the  ratings  of  the  noisiness  of  the 
computer-generated  image  showed  no  effect  of  filtering  — an  effect  that  ob- 
viously should  be  there  — we  decided  to  perform  a second  experiment  to  see  if 
more  definitive  results  could  be  obtained. 


( 
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SECTION  III 
EXPERIMENT  II 


INTRODUCTION  AND  METHODS 

The  second  experiment  was  a simplified  repetition  of  the  first  except 
there  were  some  changes  which  made  this  study  more  sensitive  to  the  effects  of 
filtering  the  signals  sent  to  the  CIG  display  processor.  For  instanct,  this 
time  subjects  familiar  with  the  ASPT  were  used.  They  were  instructor  pilots 
for  the  T-37  aircraft  who  had  between  1,000  and  4,500  hours  of  flying  time 
with  a group  average  of  over  3,200  hours.  They  had  been  assigned  to  the  Air 
Force  Human  Resources  Laboratory  to  act  as  advisors  and  subjects  for  experi- 
mental and  development  work  on  the  ASPT  and  had  "flown"  the  device  under  a 
variety  of  conditions. 

The  flying  task  was  made  easier  and  emphasis  was  placed  on  the  problems 
caused  by  flutter  of  the  visual  image.  Rather  than  have  the  lead  aircraft 
perform  climbs  and  dives,  the  formation  flight  software  was  changed  to  have 
the  lead  craft  fly  straight-and-level . To  emphasize  the  effects  of  noise 
propagated  forward  through  the  visual  system,  the  maximum  buffet  that  the  ASPT 
software  could  provide  was  added  for  the  duration  of  each  trial.  This  did 
make  controlling  the  simulator  more  difficult  than  just  flying  it  straight- 
and-level,  but  the  buffet  affected  the  translational  positions  of  both  air- 
craft similarly,  so  that  to  this  buffet,  we  added  "noise"  by  randomly  varying 
the  position  of  the  lag  aircraft's  ailerons.  This  created  differences  of 
attitude  of  the  lag  aircraft  relative  to  the  lead  that  our  pilots  had  to 
correct  in  a compensatory  manner.  This  noise  was  then  scaled  to  where  the 
instructor  pilots  agreed  that  it  plus  the  buffet  forced  the  formation  flying 
task  to  be  moderately  difficult. 

To  simplify  the  experimental  design,  the  conditions  of  nonvisual  motion 
cueing  used  in  the  first  study  were  eliminated.  The  same  filter  settings  as 
in  Experiment  1 were  tested  and  repeated  trials  were  used,  making  the  data 
collection  a simple  block-randomized  design  with  six  levels  of  a single  vari- 
able. All  six  settings  were  tested  in  each  block  of  trials  with  all  subjects 
receiving  the  same  random  order  of  conditions.  The  testing  was  done  within  a 
single  three-hour  session  of  five  six-trial  blocks. 

Piloting  control  performance  was  measured  using  the  software  developed  for 
the  first  experiment,  except  that  we  estimated  an  additional  surmary  measure. 
This  was  (see  Sterns,  1975)  — the  standard  deviation  of  a frequency  domain 

representation  of  a time  series.  It  is  a measure  easily  accumulated  in  real 
time,  and  is  proportional  to  the  bandwidth  of  a signal.  For  pilots'  control 
activities,  it  provides  an  indication  of  the  structure  of  the  changes  that 
different  settings  of  the  display  filters  encourage.  This  time  also  the  meas- 
ures of  the  x- , y-,  and  z-axis  position  were  adjusted  to  represent  error  from 
the  ideal  position  for  the  lag  aircraft. 

Again  ratings  were  used  for  the  instructor  pilots  to  quantify  their  re- 
actions to  our  changes  of  the  visual  display.  The  two  scales  developed  this 
time  were  both  nine-point  scales,  one  for  the  noise  of  the  CIG  system  and  one 
for  the  controllability  of  the  simulation.  In  constructing  these  scales. 
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attention  was  given  to  dividing  each  dimension  into  describable  sections  that 
might  enable  the  raters  to  be  consistent.  Descriptions  of  the  two  scales 
given  to  the  pilots  are  presented  in  Tables  A-5  and  A-6  of  Appendix  A.  For 
this  second  study,  more  time  was  available  for  briefing  the  pilots  so  that 
they  were  aware  of  the  nature  and  purpose  of  the  study  and  had  been  instructed 
on  the  use  of  the  rating  scales.  Some  prot'ems  of  interpretation  occurred  as 
testing  started,  so  we  used  the  first  block  of  six  trials  to  indicate  to  the 
pilots  what  we  felt  were  the  worst  and  bes:  cases  of  each  dimension. 

To  remove  the  problem  of  edges  being  deleted  during  a flight,  the  ASPT  CIG 
system  was  operated  at  75  percent  of  full  capacity  this  time.  The  result  was 
an  image  of  the  lead  aircraft  without  all  of  the  detail  of  the  ones  used 
in  Experiment  I,  but  with  all  of  the  cues  shown  in  Figure  2 that  pilots  use  to 
maintain  the  fingertip  position.  As  in  the  first  experiment,  the  first  20 
seconds  after  the  device  was  activated  (but  before  data  were  collected)  were 
used  for  the  pilot  to  recover  the  fingertip  position,  and  in  this  study  the 
display  filters  were  not  active  till  the  end  of  that  period.  We  felt  that 
allowing  the  pilots  an  interval  to  observe  the  no-filtering  condition  before 
each  trial  would  also  help  to  make  their  ratings  more  stable.  To  that  end, 
the  instructor  pilots  were  asked  to  observe  this  "worst"  case  of  jitter  of  the 
visual  image  and  use  it  for  comparison  to  the  behavior  of  the  display  during 
the  rest  of  the  trial.  During  the  pilots'  briefing,  they  were  reminded  that 
at  the  altitude  and  airspeed  of  this  simulation,  the  longitudinal  axis  of  the 
T-37  aircraft  was  more  responsive  than  the  lateral  one  and  that  the  greatest 
jitter  would  appear  on  the  horizon  line  directly  ahead.  When  flying  in  the 
fingertip  position,  the  pilots  attended  to  the  position  of  the  lead  aircraft 
to  their  left,  so  as  part  of  the  experimental  instructions,  they  were  asked 
to  notice  the  behavior  of  the  forward  horizon  and  to  take  this  into  account 
as  they  rated  the  jitter  of  the  display. 

Because  of  the  subject  and  trials  effects  of  Experiment.  1,  differences 
attributable  to  those  sources  were  removed  by  expressing  eacr.  measure  col*- 
lected  within  a block  of  six  trials  as  a z-score  — l.e.,  the  six  observa- 
tions of  each  measure  were  normalized,  forcing  the  effects  due  to  different 
subjects  and  blocks  of  trials  to  be  zero  and  leaving  the  effect  due  to 
different  settings  of  the  display  filters  as  the  only  one  analyzable.  This 
effect  was  then  tested  with  a simple  one-way  analysis  of  variance  performed 
on  the  subjects'  average  performances.  The  means  of  repeated  trials  were 
used  in  these  analyses  so  that  the  findings  of  this  experiment  apply  only  to 
average  performance  and  do  not  reflect  typical  trial-by-trial  variability. 

RESULTS  AND  DISCUSSION 

Tables  B-16  to  B-26  of  Appendix  B present  the  results  of  the  analyses  of 
variance  performed  on  the  objective  measures  of  this  second  study.  Each 
table  presents  the  one-way  analysis  of  the  average  value,  variance,  and 
bandwidth  for  a given  measure.  This  time  n2  for  all  significant  effects  was 
0.66.  Averaging  repeated  trials  also  reduces  the  error  variance  of  experi- 
ments — somewhat  unfairly  in  the  sense  that  information  about  subjects' 
trial -by-trial  variability  is  lost  — but  in  this  case  the  Interest  was  in 
device  design,  not  human  performance. 
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OBJECTIVE  MEASURES.  The  results  of  this  experiment  can  be  divided  into  three 
categories:  those  most  likely  related  to  our  filtering  (in  general)  of  the 
signals  sent  to  the  CIG  system,  those  related  to  our  filtering  in  the  context 
of  the  formation  flight  task,  and  those  that  seem  a bit  arbitrary  and  that 
probably  are  spurious.  Effects  in  the  first  category  are  changes  of  the  pitch 
and  roll  angles  of  the  lag  aircraft  and  the  changes  of  piloting  control  that 
produced  them.  As  the  display  was  made  more  sensitive  by  raising  the  break 
frequency  of  the  filters,  the  pilots  responded  by  controlling  their  aircraft 
so  as  to  reduce  the  variance  of  its  pitch  and  roll  angles  and  to  change  the 
bandwidth  toward  its  value  for  the  no-filtering  condition.  This  required  an 
increase  of  bandwidth  along  the  aircraft's  longitudinal  axis  and  a slight  re- 
duction along  the  lateral  one.  These  functions  are  displayed  in  Figures  10 
and  11.  These  changes  of  aircraft  response  were  accomplished  by  parallel 
changes  of  the  pilots'  use  of  their  control  stick.  On  both  the  longitudinal 
and  lateral  axes,  pilots  reduced  the  variance  of  the  position  of  the  control 
stick,  shown  in  Figure  12,  and  Its  bandwidth,  shown  In  Figure  13.  Their  use 
of  the  throttles  also  changed  as  a function  of  the  filtering  break  frequency, 
paralleling  the  pattern  of  results  seen  In  their  use  of  the  control  stick.  As 
the  setting  of  the  filter  allowed  more  high  frequency  information  to  the  dis- 
play, lower  average  settings  for  the  throttles  were  used,  and  the  variance  of 
these  settings  was  reduced.  These  trends  are  shown  in  Figures  14  and  15.  As 
in  Experiment  1,  the  variance  of  a measure  seemed  the  datum  most  likely  to  be 
affected  by  our  manipulations,  a finding  similar  to  that  of  Cooper,  Harris, 
and  Sharkey  (1975).  Clearly  when  the  display  was  filtered  strongly,  so  that 
it  appears  too  sluggish,  our  pilots  worked  more  to  control  the  simulated  air- 
craft. Contrary  to  the  data  of  Experiment  I,  there  is  no  indication  that  the 
pilots  will  control  more  poorly  without  the  filtering  than  with  it.  This  may 
reflect  our  use  of  pilots  familicir  with  a given  device  or  merely  human  con- 
trollers' willingness  to  ignore  the  annoying  high  frequency  activity  in  an 
unfiltered  display. 

Changes  of  aircraft  position  and  of  related  piloting  control  that  were 
placed  into  the  second  category  were  similar  to  some  seen  in  Experiment  I. 
These  were  the  reduction  of  the  variance  of  the  difference  of  the  yaw  angle 
of  the  lag  aircraft  presented  in  Figure  16,  and  the  reduction  of  the  variance 
of  the  rudder  pedal  position  shown  in  Figure  17.  These  changes  probably 
reflect  the  tactic  the  instructor  pilots  stated  they  used  to  fly  formation 
during  our  simulation  — which  was  that  they  would  "crab"  into  the  fingertip 
position  and  then  maintain  a yaw  angle  to  counter  the  effect  of  the  lead  air- 
craft's backwash.  Whenever  a large  gust  upset  the  lag  aircraft,  the  pilots' 
response  was  to  move  laterally  (to  the  right)  and  then  carefully  maneuver 
back  into  position.  It  was  probably  these  responses  that  caused  significant 
effects  in  the  variances  of  the  yaw  angle  and  rudder  pedal  positions. 

Another  flying  task,  like  aerial  refueling  or  air-to-ground  weapons  delivery, 
may  not  show  similar  effects  on  these  measures. 

Probably  related  to  the  above  technique  was  a tendency  to  keep  the  aver- 
age position  of  the  control  stick  a bit  off-center  (to  the  right)  that  was 
accentuated  at  low  break  frequencies  of  the  display  filters.  This  was 
probably  used  to  counteract  the  tendency  of  the  backwash  to  roll  the  lag 
aircraft  to  the  left,  and  when  the  display  was  sluggish  (relatively),  the 
instructor  pilots  were  conservative  and  tended  to  keep  their  craft  prepared 
to  roll  to  the  right.  This  use  of  the  control  stick  is  documented  in  Figure 
18. 
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Filter  Break  Frequency  in  Hz 


Figure  10.  Variance  of  the  difference  of  pitch  and  roll  angle  of 
the  lead  and  lag  aircraft  as  a function  of  filter 
setting. 
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Filter  Break  Frequency  in  Hz 

Figure  11.  Bandwidth  of  the  pitch  and  roll  angle  differences  of  the 
lead  and  lag  aircraft  as  a function  of  filter  setting. 
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Filter  Break  Freouencv  in  Hz 


Figure  12.  Variance  of  the  pilots'  longitudinal  and  lateral-axis 
movements  of  the  control  stick  as  a function  of  filter 
setting. 


Figure  13.  Ban&ridth  of  the  pilots'  longitudinal  and  lateral-axis 
movements  of  the  control  stick  as  a function  of  filter 
setting. 
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Filter  Break  Frequency  in  Kz 

Figure  14.  Average  position  of  left  and  right  throttles  as  a 
function  of  filter  setting. 
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Filter  Break  Frequency  in  Hz 

Figure  16.  Variance  of  the  differences  of  yaw  angle  of  the  lead  and 
lag  aircraft  as  a function  of  filter  setting. 


Filter  Break  Frequency  in  Hz 

Figure  17.  Variance  of  the  position  of  the  rudder  pedals  as  a function 
of  filter  setting. 
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Filter  Break  Frequency  in  Hz 

Figure  18.  Average  lateral  position  of  the  control  stick  as  a function 
of  filter  setting. 


More  firmly  in  the  odds  and  ends  category  are  the  changes  of  the  average 
z-axis  position  and  its  variance  shown  in  Figures  19  and  20.  Our  instructor 
pilots  tended  to  fly  the  ASPT  simulation  a little  too  high  at  low  filter  set- 
tings, and  this  height  along  with  its  variance  was  reduced  as  the  filter  set- 
ting was  increased  to  1 Hz.  Again,  little  change  was  seen  beyond  that  point. 
The  z-axis  information  to  the  CIG  display  system  was  filtered  because  of 
peculiarities  of  the  simulation  of  formation  flight  in  the  ASPT.  Undoubtedly 
this  effect  on  the  measure  of  position  along  the  z-axis  reflects  this  aspect 
of  our  simulation. 

Finally,  Figure  21  presents  the  bandwidth  changes  seen  in  the  x-axis 
difference  of  position  of  the  two  aircraft.  Unless  it  reflects  the  pilots' 
changes  of  their  use  of  the  aircraft's  throttles,  we  do  not  know  why  this 
measure  showed  the  effect  that  it  did.  Statistically  the  effect  is  quite 
large,  but  the  spectral  content  of  this  particular  time-series  is  quite  low 
frequency,  so  the  changes  of  position  may  well  reflect  the  changes  of  engine 
thrust  used  by  the  pilots. 

RATING  MEASURES.  The  subjective  ratings  collected  this  time  were  more  useful 
as  they  consistently  differed  with  changes  of  the  filter  settings.  The 
ratings  of  both  the  flutter  in  the  visual  display  and  the  controllability  of 
the  simulation  fit  the  one-way  analysis-of-variance  design,  and  after  the 
ratings  were  converted  to  z-scores,  they  were  so  analyzed.  While  the  ratings 
of  noise  in  the  CIG  display  were  made  on  a nine-point  scale  where  high 
numbers  represented  a stable  display  and  low  numbers  a jittery  one,  the 
ratings  of  flight  control  were  made  on  a scale  with  the  most  controllable 
point  at  its  center.  The  extremes  of  this  scale  — high  or  low  numbers  — 
were  used  for  aircraft  responses  that  seemed  too  sluggish  or  too  responsive. 
Our  pilots  had  great  difficulty  assigning  consistent  ratings  to  the  flights 
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Fil ter  Break  Frequency  in  Hz 

Figure  19.  Average  difference  of  z-axls  position  of  the  lead  and 
lag  aircraft  as  a function  of  filter  setting. 


Filter  Break  Frequency  in  Hz 

Figure  20.  Variance  of  the  z-axls  difference  of  position  of  the 
lead  and  lag  aircraft  as  a function  of  filter  setting. 
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Figure  21.  Bandwidth  of  the  x-axIs  difference  of  position  of  the  lead 
and  lag  aircraft  as  a function  of  filter  setting. 
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where  the  filter  was  set  to  a break  frequency  of  1/4  or  1/2  Hz.  Particular 
events  during  those  trials  seemed  to  be  selected  for  the  rating  and  the  result 
was  that  the  ratings  for  these  low-frequency  settings  of  the  filters  were 
quite  variable.  The  pilots  pilots  agreed  that  those  settings  were  not  desir- 
able, but  they  could  not  consistently  extract  from  the  behavior  of  the  display 
just  what  it  was  that  they  did  not  like.  Occasionally  a gust  would  sum  with  a 
control  input  and  the  condition  was  judged  too  responsive,  and  just  as  fre- 
quently, the  same  filter  setting  was  judged  too  sluggish.  So  to  remove  this 
ambiguity,  the  ratings  of  flight  controllability  were  "folded"  to  become  a 
one-to-five  scale  (ratings  of  six,  seven,  eight,  and  nine  became  ratings  of 
four,  three,  two  and  one).  Now  high  numbers  on  the  scale  represent  control 
that  was  easy  and  low  numbers  represent  difficult  control,  regardless  of  the 
reason. 

The  results  of  the  analyses  performed  on  the  rating  data  are  presented  in 
Table  B -27  of  Appendix  B and  are  depicted  in  Figure  22.  The  pilots  judged  the 
flights  to  be  significantly  less  well  controlled  under  the  1/4  Hz  setting, 
with  the  other  conditions  judged  about  equally  well  controlled,  but  perhaps 
most  significant  in  a practical  sense,  their  judgments  of  display  jitter  show 
a monotonic  decrease  with  filter  setting  with  the  no-filtering  condition  felt 
to  be  the  worst  case.  Although  the  relative  scaling  of  the  two  ratings  is 
immaterial,  we  can  see  that  a mid-range  of  filter  settings  is  preferable  to 
either  a low-frequency  setting  or  a no-filtering  condition.  Their  ratings 
of  system  controllability  parallel  objective  measures  of  flight  control  — 
i.e.,  as  the  pilots  changed  their  control  activity  so  that  the  variance  or 
bandwidth  of  a measure  approached  its  value  under  the  no-filtering  condition, 
the  controllability  of  the  simulation  was  felt  to  increase,  while  at  the  same 
time  they  judged  the  jitter  of  the  computer-generated  visual  Image  to  become 
more  and  more  annoying  with  the  unfiltered  image  judged  as  containing  the  most 
noise. 
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Filter  Dreak  Frequency  in  Hz 

Figure  22.  Pilot  ratings  of  flight  control  and  CIG  system 
noise  as  a function  of  filter  setting. 


So  that  the  main  finding  of  this  study  can  be  displayed  more  clearly,  we 
have  plotted  in  Figure  23  the  normalized  ratings  of  the  noisiness  of  the 
visual  image  and  the  average  of  the  normalized  pitch  and  roll  errors  --  both 
as  a function  of  the  break  frequency  of  the  display  signal  filters.  Both  sets 
of  data  are  expressed  as  z-scores  to  indicate  where  a particular  measure  was 
above  or  below  its  experimental  average  of  zero.  Here  we  can  see  that  as  the 
break  frequency  of  the  filters  was  raised,  aircraft  control  became  more  and 
more  accurate  until  a “best"  setting  of  1 Hz  was  reached,  while  at  the  same 
time,  the  ratings  of  the  visual  display  were  not  drastically  reduced  until  a 
setting  above  1 Hz  was  reached.  Clearly  a relatively  "safe"  setting  for  the 
filters  can  be  defined  where  the  changes  to  the  CIG  system's  inputs  will  not 
degrade  piloting  control  and  pilots  will  feel  the  display  has  acceptable 
stability.  This  is  the  shaded  range  in  the  figure. 

We  feel  that  some  factors  of  flight  simulations  should  determine  where  the 
break  point  of  display  filters  should  be.  Should  this  sort  of  delay  compensa- 
tion be  used  for  a simulation  of  air-to-air  combat  using  high  performance  air- 
craft, a setting  near  the  upper  end  of  the  safe  range  would  be  preferred,  and 
conversely,  a task  that  requires  the  display  of  only  low  rates  of  angular 
acceleration  may  well  benefit  by  using  a low  frequency  setting.  Along  with 
the  sort  of  flying  task,  the  type  of  aircraft  simulated  will  determine  the 
filter  setting,  as  obviously,  a simulation  of  an  aircraft  that  can  have  high 
frequency  components  In  its  responses  should  have  those  components  reach  a 
visual  display.  The  trade  that  filtering  allows  is  one  of  preference  for  a 
particular  amount  of  display  stability  vs.  flying  performance.  For  the  T-37 
aircraft  and  the  simulation  of  formation  flight  used  for  this  study,  there 
clearly  was  a range  of  settings  that  allowed  the  degree  of  flight  control 
normally  found  in  the  ASPT  and  that  also  removed  most  of  the  annoying  jitter 
in  the  CIG  system's  Image.  From  these  two  experiments,  we  suggest  that  this 
range  is  from  3/4  to  1 Hz. 
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1/4  1/2  V4  1 2 

Filter  Break  Frequency  in  Hz 


Figure  23.  Average  flight  control  and  opinion  of  the  computer- 
generated image  as  a function  of  filter  setting.  The 
measure  of  flight  control  Is  the  average  of  the  variances 
of  the  differences  of  pitch  and  roll  angle,  and  both 
sets  of  data  are  normalized. 
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SECTION  IV 


CONCLUSIONS 

a.  The  predict-then-filter  scheme  implemented  on  the  ASPT  is  useful  for 
delay  compensation  in  operational  flight  simulators  that  have  significant 
transport  delays  in  the  operation  of  their  CIG  visual  systems. 

b.  The  usefulness  of  the  scheme  is  that  a filter  setting  can  be  chosen 
that  reduces  the  annoying  high  frequency  components  of  the  signals  sent  to 
the  visual  display  processor  without  affecting  flying  control  performance, 
and  that  it  can  easily  be  implemented  as  either  hardware  or  computer  software. 

c.  The  scheme  affects  piloting  control  only  when  useful  low  frequency 
information  (below  1/2  Hz)  was  removed  from  the  CIG  display.  Tasks  other  than 
formation  flight  may  require  higher  frequency  information,  hut  many  of  the  fly- 
ing tasks  for  which  CIG  displays  are  used  probably  have  requirements  similar 

to  those  of  flying  formation. 

d.  For  a given  amount  of  lead,  it  would  be  relatively  easy  to  tailor  this 
scheme  for  a particular  task  or  device  by  manipulating  a single  parameter  — 
the  break  frequency  of  the  display  filter. 

e.  For  most  of  the  parameters  of  aircraft  control  and  pilot  input  that 
we  observed,  the  variance  was  the  measure  most  sensitive  to  changes  of  the 
visual  display.  In  Experiment  II,  the  bandwidth  measure  also  was  sensitive 
to  our  manipulations  and  seemed  to  correlate  with  the  variance  — in  that 
changes  of  bandwidth  were  probably  the  activity  reflected  as  significant 
differences  of  variance. 
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APPENDIX  A 

FORMATION  FLIGHT  GRADING  CRITERIA 

TABLE  A-l.  12-POINT  SCALE  USED  FOR  RATING  FLIGHT  CONTROL  IN  EXPERIMENT  I 
EXCELLENT 

1.  Excellent  is  characterized  by  exceptionally  smooth  control  and  small  devia- 
tions from  the  desired  position.  Excellent  position  is  maintained  within 
deviations  as  follows: 

a.  Fore-Aft  (longitudinal):  +2  ft. 

b.  Side-side  (lateral):  from  1 ft  closer  in  (tighter)  to  3 ft  further 

out. 


c.  Vertical:  from  less  than  1 ft  high  to  not  more  than  3 ft  low. 

2.  Good.  Good  is  characterized  by  smooth  control,  with  larger  deviations  and 
longer  time  out  of  position.  Deviations  must  range  within: 

a.  Fore-aft  (longitudinal)  +3  ft. 

b.  Side-side  (lateral)  from  1 ft  closer  in  (tighter)  to  5 ft  further  out. 

c.  Vertical:  from  2 ft  high  to  4 ft  low. 

3.  Fair.  Fair  is  characterized  by  rough  control  with  large  deviations,  al- 
most always  out  of  position.  Deviations  range  within: 

a.  Fore-aft  (longitudinal)  i 5 ft 

b.  Side-side  (lateral)  from  2 ft  closer  in  (tighter)  to  8 ft  further  out. 

c.  Vertical:  from  4 ft  high  to  6 ft  low. 

4.  Unsatisfactory.  Unsatisfactory  i-  characterized  by  the  Inability  to  main- 
tain even  fair  position.  Aircraft  control  is  rough. 


TABLE  A-2.  DESCRIPTION  OF  12-POINT  SCALE  VALUES  OF  TABLE  A-l  USED  FOR 
EXPERIMENT  I 


EXAMPLE  GRADES 

12  Perfect  position  maintained,  control  use  smooth,  accurate. 

11  Only  small  deviations  seen.  Control  use  smooth.  Errors  quickly  corrected. 

10  Deviated  from  desired  only  within  the  excellent  area;  however,  did  not 
remain  "on"  desired  position,  but  passed  through  it. 

9 Remained  tightly  within  the  "good"  area.  Control  use  smooth.  Nearly  an 
excellent. 
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TABLE  A-2.  DESCRIPTION  OF  12-POINT  SCALE  VALUES  OF  TABLE  A-1  USED  FOR 
EXPERIMENT  I (Cont'd) 

8 A solid  good.  Normal  use  of  controls.  Was  in  position  most  of  the  time. 

7 Stayed  in  good  area,  but  control  use  was  a little  rough.  Drifted  con- 
stantly out  of  position. 

6 Nearly  a "good,"  but  deviations  somewhat  too  large.  Correction  ti.ne  too 
slow,  and  control  use  too  rough. 

5 Kept  aircraft  in  fair  position  but  was  constantly  jockeying  the  controls. 
Almost  always  correcting  errors.  Somewhat  behind  the  aircraft. 

4 Barely  stayed  in  the  basic  formation  flight  area.  Aircraft  control  poor. 
Almost  unsafe. 

3 Able  to  move  the  aircraft  Into  the  "fair"  area,  but  unable  to  keep  it 
there.  Constmtly  "behind"  the  other  aircraft. 

2 Did  not  crash  or  otherwise  jeopardize  life,  but  could  not  maneuver  into 
position. 

1 Crashed,  passed  under,  over,  or  fell  out  of  range. 


TABLE  A-3.  DESCRIPTION  OF  CIG  RATING  SCALE  OF  EXPERIMENT  I 
ADDITIONAL  RATING  INFORMATION 

We  expect  that  some  of  the  computer  software  changes  that  we  will  make 
between  trials  will  affect  the  characteristics  of  the  CIG  visual  display,  and 
therefore  after  each  trial,  we  would  like  you  to  rank  the  display's  "goodness" 
or  usefulness  for  formation  flight.  This  will  be  in  addition  to  the  numerical 
score  you  will  give  the  flight,  and  in  order  to  avoid  confusion,  we  would  like 
you  to  use  a letter  scale  where  A = good,  B = average,  and  C = poor,  and  where 
pluses  and  minuses  are  allowed.  The  ratings  can  thus  go  from  A+  to  C-,  and 
for  decisions  like  B-  vs  C+,  you  will  have  to  do  the  best  you  can. 

These  letter  scores  can  be  entered  on  the  flight  scoresheet  after  the 
numerical  score  for  a given  trial,  and  the  first  entry  miqht  look  like: 

1 6,  C+  . 
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TABLE  A-4. 
IP  or  OBS 


S 2 

O 

r~ 

=*= 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

SCORE  SHEET  USED  BY  INSTRUCTOR  PILOTS  IN  EXPERIMENT  I 

NAME 

DATE 


GRADING: 


- U + 

- F + 

-TT+ 

- £ + 

1 2 3 

4 5 6 

7 § 5" 

10  11  12 

OVER  4 

4 

2 

1 HIGH 

OVER  6 

6 

4 

3 LOW 
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5 

3 

2 FORWARD 
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5 

3 

2 BACK 

OVER  2 

2 

1 

1 LEFT 

OVER  8 

8 

5 

3 RIGHT 
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TABLE  A-5.  RATING  SCALE  USED  TO  JUDGE  AIRCRAFT  CONTROL  IN  EXPERIMENT  II 
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Barely  usable,  very  annoying  jitter  Poor 
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APPENOIX  B 

ANALYSIS  OF  VARIANCE  TABLES  OF  EXPERIMENTS  I AND  II 

These  tables  report  the  analyses  of  variance  of  Experiments  I and  II. 

In  them,  the  exact  probability  of.,F-ratios  significant  at  the  p<.05  level 
or  greater  are  given  along  with  n*  “ The  proportion  of  the  total  sum  of 
squares  attributable  to  each  significant  effect.  The  analyses  for  a riven 
measure  are  grouped  together  — those  of  the  average  value  and  variance  for 
the  data  of  Experiment  I and  of  the  average  value,  variance,  and  bandwidth 
for  Experiment  II.  Occasionally  rather  large  numbers  were  encountered  as 
sums  of  squares  and  to  conserve  space,  these  were  expressed  in  exponential  or 
scientific  notation.  The  value  to  four  places  is  expressed  as  a decimal  and 
the  number  following  the  E is  an  exponent  of  ten.  For  example,  0.1234E7, 

would  represent  0.1234  x 107  or  1,234,000. 


TABLE  B-l . ANALYSIS  OF  THE  AVERAGE  VALUE  (A)  AND  VARIANCE  (B)  OF  THE 
X-AXIS  DIFFERENCE  OF  POSITION  OF  THE  LEAD  AND  LAG  AIRCRAFT 


(A) 


SOURCE 

SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

P n2 

Blocks 

169294.60 

3 

56431 .53 

0.42 

Treatments 

7205487.50 

53 

135952.59 

1.01 

A 

255036.60 

2 

127918.30 

0.96 

B 

659077.61 

5 

131815.52 

0.98 

C 

191216.69 

2 

95608.34 

0.71 

AxB 

1233427.90 

10 

123342.79 

0.92 

AxC 

532276.81 

4 

133069.20 

0.99 

BxC 

1638471.70 

10 

163847.17 

1.22 

AxBxC 

2695180.30 

20 

134759.02 

1.00 

Residual 

Total 

2138S  20.00 

2575 -TOT 

159 

2TET 

134526.54 

.744 

(B) 

Blocks 

.12936E+11 

3 

.43122E+10 

1.05 

Treatments 

.221 96E+12 

53 

.41880E+10 

1.02 

A 

.78885E+10 

2 

.39442E+10 

0.96 

B 

.52143E+1 1 

5 

.10428E+11 

2.55 

.0295  .058 

C 

. 1 7582E+1 1 

2 

.87914E+10 

2.15 

AxB 

.40799E+11 

10 

.40799E+10 

0.99 

AxC 

.17651E+10 

4 

.44129E+9 

0.11 

BxC 

.61820E+11 

10 

.61820E+10 

1.51 

AxBxC 

.39965E+1 1 

20 

.19982E+10 

0.49 

Residual 

Total 

.65047E+1 2 
:55537T+10 

159 

2TT 

.40910E+10 

.735 
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TABLE  B-2.  ANALYSIS  OF  THE  AVERAGE  VALUE  (A)  AND  VARIANCE  (B)  OF  THE 
Y-AXIS  DIFFERENCE  OF  POSITION  OF  THE  LEAD  AND  LAG  AIRCRAFT 


(A) 


SOURCE 

SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

JL_ 

n2 

Blocks 

1624517.20 

3 

541505.73 

1.64 

Treatments 

9734820.40 

53 

183675.86 

0.55 

A 

437389.08 

2 

218694.54 

0.66 

B 

1657612.60 

5 

331522.53 

1.00 

C 

932172.32 

2 

466086.16 

1.41 

AxB 

3687199.70 

10 

368719.97 

1.11 

AxC 

1375824.70 

4 

343956.18 

1.04 

BxC 

3264811.30 

10 

326481.13 

0.99 

AxBxC 

6378910.40 

20 

318990.52 

0.96 

Residual 

52635920.00 

159 

331043.52 

.731 

Total 

7TW3257.0 

277 

(B) 

Blocks 

.32476E+9 

3 

.10825E+9 

0.47 

Treatments 

.11274E+1 2 

53 

.21272E+10 

0.93 

A 

.31673E+10 

2 

.T5836E+10 

0.70 

B 

•13551E+11 

5 

.27102E+10 

1.19 

C 

.19453E+10 

2 

.97268E+9 

0.43 

AxB 

.28850E+11 

10 

.28850E+10 

1.27 

AxC 

.10965E+11 

4 

.27413E+10 

1.20 

BxC 

.23679E+11 

10 

.23679E+10 

1.04 

AxBxC 

.30584E+11 

20 

.15292E+10 

0.67 

Residual 

.36109E+12 

159 

.22710E+10 

.757 

Total 

.47709E+T2 

217 

TABLE  B- 

-3.  ANALYSIS  OF  THE  AVERAGE  VALUE  (A) 

AND  VARIANCE  (B) 

OF  THE 

Z-AXIS  DIFFERENCE  OF  POSITION  OF 

THE  LEAD  AND  LAG 

AIRCRAFT 

(A) 

SOURCE 

SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

_e_ 

n2 

Blocks 

3035.96 

3 

1011.99 

5.79 

.0012 

.076 

Treatments 

9298.42 

53 

175.44 

1.00 

A 

669.17 

2 

334.59 

1.91 

B 

728.01 

5 

145.60 

0.83 

C 

206.71 

2 

103.36 

0.59 

AxB 

2106.55 

10 

210.65 

1.20 

AxC 

1157.10 

4 

289.28 

1.65 

BxC 

1632.07 

10 

163.21 

0.93 

AxBxC 

2798.80 

20 

139.94 

0.80 

Residual 

27807.77 

159 

174.89 

.693 

Total 

40147.15 

217 
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TABLE  B-3.  ANALYSIS  OF  THE  AVERAGE  VALUE  (A)  AND  VARIANCE  (B)  OF  THE  Z-AXIS 
DIFFERENCE  OF  POSITION  OF  THE  LEAD  AND  LAG  AIRCRAFT  (Cont'd) 


(B) 

SOURCE 

SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO  p 

n2 

Blocks 

. 34581 E+10 

3 

.1 1527E+10 

1.59 

Treatments 

.29356E+1 1 

53 

.55389E+9 

0.76 

A 

. 1 0544E+1 0 

2 

. 52723  E+9 

0.73 

B 

.46242E+10 

5 

.92485E+9 

1.28 

C 

.27301  E+10 

2 

.1 3650E+10 

1.88 

AxB 

.69328E+10 

10 

.69328E+9 

0.96 

AxC 

.27469E+10 

4 

.68675E+9 

0.95 

BxC 

.81067E+10 

10 

.81067E+9 

1.12 

AxBxC 

.13161  E+ll 

20 

.65804E+9 

0.91 

Residual 

.115321+1 2 

159 

.72527E+9 

.729 

Total 

T753T3E+77 

m 

TABLE  B- 

■4.  ANALYSIS  OF  THE  AVERAGE  VALUE  (A)  AND  VARIANCE 

(B)  OF  THE 

DIFFERENCE  OF  ROLL  ANGLE  OF  THE  LEAD  AND  LAG  AIRCRAFT 


(A) 


SOURCE 

SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

_e_  _ni_ 

Blocks 

93.99 

3 

31.33 

2.98 

.0325  .039 

Treatments 

636.12 

53 

12.00 

1.14 

A 

8.29 

2 

4.14 

0.39 

B 

35.50 

5 

7.10 

0.67 

C 

80.72 

2 

40.36 

3.83 

.0230  .034 

AxB 

137.90 

10 

13.79 

1.31 

AxC 

133.00 

4 

33.25 

3.16 

.01 56  .055 

BxC 

100.57 

10 

10.06 

0.96 

AxBxC 

140.13 

20 

7.01 

0.67 

Residual 

1673.88 

159 

10.53 

.696 

Total 

2403.59 

2TT 

(B) 


Blocks 

193812.92 

3 

64604.31 

3.55 

.0157 

.045 

Treatments 

1199290.78 

53 

22628.13 

1.24 

A 

42240.93 

2 

21120.47 

1.16 

B 

65286.83 

5 

13057.37 

0.72 

C 

215488.60 

2 

107744.30 

5.92 

.0031 

.050 

AxB 

215077.45 

10 

21507.75 

1.18 

AxC 

63066.01 

4 

15766.50 

0.87 

BxC 

162226.06 

10 

16222.61 

0.89 

AxBxC 

435904.88 

20 

21795.24 

1.20 

.675 

Residual 

2893975.20 

159 

18201.10 

Total 

4287078.80 

m 
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TABLE  B-5.  ANALYSIS  OF  THE  AVERAGE  VALUE  (A)  AND  VARIANCE  (B)  OF  THE 


DIFFERENCE  OF 

PITCH  ANGLE  OF  THE 

LEAD  AND 

LAG  AIRCRAFT 

(A) 

SOURCE 

SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

. _e_  _ni_ 

Blocks 

0.84 

3 

0.28 

1.14 

Treatments 

13.60 

53 

0.26 

1.04 

A 

0.42 

2 

0.21 

0.85 

B 

1.13 

5 

0.23 

0.92 

C 

0.29 

2 

0.15 

0.59 

AxB 

2.91 

10 

0.2S 

1.18 

AxC 

1.54 

4 

0.38 

1.56 

BxC 

3.33 

10 

0.33 

1.35 

AxBxC 

3.98 

20 

0.20 

0.81 

Residual 

39.13 

159 

0.25 

.731 

Total 

53.56 

m 

(B) 

Blocks 

739.28 

3 

246.43 

12.83 

.0000  .130 

Treatments 

1884.06 

53 

35.55 

1.85 

.0021  .332 

A 

62.02 

2 

31.01 

1.61 

B 

294.63 

5 

58.93 

3.07 

.0114  .052 

C 

641.73 

2 

320.86 

16.70 

.0000  .113 

AxB 

269.75 

10 

26.97 

1.40 

AxC 

39.42 

4 

9.86 

0.51 

BxC 

216.05 

10 

21.61 

1.12 

AxBxC 

360.45 

20 

18.02 

0.94 

Residual 

3054.71 

159 

19.21 

.538 

Total 

5578.04 

?nr 

TABLE  B-6.  ANALYSIS  OF  THE  AVERAGE  VALUE  (A)  AND  VARIANCE  (B)  OF  THE 
DIFFERENCE  OF  YAW  ANGLE  OF  THE  LEAD  AND  LAG  AIRCRAFT 


(A) 

SOURCE 

SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

Bl ocks 

203.00 

3 

67.67 

0.24 

Treatments 

17404.79 

53 

328.39 

1.18 

A 

206.74 

2 

103.37 

0.37 

B 

1666.96 

5 

333.39 

1.19 

C 

920.13 

2 

460.07 

1.65 

AxB 

3464.58 

10 

346.46 

1.24 

AxC 

1692.22 

4 

423.05 

1.51 

BxC 

3123.97 

10 

312.40 

1.12 

AxBxC 

6330.19 

20 

316.51 

1.13 

Residual 

44424.89 

159 

279.40 

Total 

62032.68 
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TABLE  B-6.  ANALYSIS  OF  THE  AVERAGE  VALUE  (A)  AND  VARIANCE  (B)  OF  THE 

DIFFERENCE  OF  YAW  ANGLE  OF  THE  LEAD  AND  LAG  AIRCRAFT  (Cont'd) 


(B) 

SOURCE 

SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

n2 

Blocks 

.18490E+8 

3 

61634E+7 

1.92 

Treatments 

.2841 9E+9 

53 

53620E+7 

1.68 

.0015 

.034 

A 

.291 95E+8 

2 

14597E+8 

4.56 

.0118 

.027 

B 

.22152E+8 

5 

44304 E+7 

1.38 

C 

.18699E+8 

2 

93496E+7 

2.92 

AxB 

.4001 3E+8 

10 

4001 4 E+7 

1.25 

AxC 

.11393E+8 

4 

28481 E+7 

0.89 

BxC 

.42749E+8 

10 

42749E+7 

1.34 

AxBxC 

•11998E+9 

20 

5 99  93 E+7 

1.87 

.0178 

.148 

Residual 

.50366E+9 

15* 

31 991  E+7 

.627 

Total 

.B1T34E+9 

m 

TABLE  B-7.  ANALYSIS  OF  THE  AVERAGE  VALUE  (A)  AND  VARIANCE  (B)  OF  THE 


LATERAL  MOVEMENTS 

OF  THE  CONTROL' 

STICK 

■ 

(A) 

SOURCE 

SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

P n2 

Blocks 

0.038 

3 

0.013 

6.38 

.0007  .077 

Treatments 

0.145 

53 

0.003 

1.36 

A 

0.009 

2 

0.004 

2.16 

B 

0.016 

5 

0.003 

1.62 

C 

0.059 

2 

0.029 

15.03 

.0000  .119 

AxB 

0.016 

10 

0.002 

0.79 

AxC 

0.008 

4 

0.002 

1.07 

BxC 

0.006 

10 

0.001 

0.31 

AxBxC 

0.031 

20 

0.002 

0.80 

Residual 

0.312 

159 

0.002 

.632 

Total 

0.494 

7TT 

(B) 

Blocks 

13.12 

3 

4.37 

7.99 

.0002  .083 

Treatments 

58.01 

53 

1.09 

2.00 

.0007  .367 

A 

3.20 

2 

1.60 

2.93 

B 

12.30 

5 

2.46 

4.49 

.0010  .078 

C 

26.48 

2 

13.24 

24.19 

.0000  .167 

AxB 

4.73 

10 

0.47 

0.86 

AxC 

0.62 

4 

0.15 

0.28 

BxC 

5.70 

10 

0.57 

1.04 

AxBxC 

4.99 

20 

0.25 

0.46 

Residual 

87.01 

159 

0.55 

.550 

Total 

T5S7PT 

2TT 
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TABLE  B-8.  ANALYSIS  OF  THE  AVERAGE  VALUE  (A)  AND  VARIANCE  (B)  OF 
THE  LONGITUDINAL  MOVEMENTS  OF  THE  CONTROL  STICK 


(A) 

SOURCE 

SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

n2 

Blocks 

0.0024 

3 

0.0008 

0.08 

Treatments 

0.3740 

53 

0.0070 

0.70 

A 

0.0032 

2 

0.0016 

0.16 

B 

0.0367 

5 

0.0073 

0.73 

C 

0.0003 

2 

0.0001 

0.01 

AxB 

0.0852 

10 

0.0085 

0.84 

AxC 

0.0173 

4 

0.0043 

0.43 

BxC 

0.0650 

10 

0.0065 

0.64 

AxBxC 

0.1683 

20 

0.0084 

0.83 

Residual 

1 .6046 

159 

0.0101 

.809 

Total 

m 

(B) 

Blocks 

7.35 

3 

3.45 

14.68 

.0000 

.152 

Treatments 

14.61 

53 

0.28 

1.65 

.0094 

.301 

A 

0.48 

2 

0.24 

1.43 

B 

2.40 

5 

0.48 

2.88 

.0161 

.050 

C 

7.91 

2 

3.96 

23.72 

.0000 

.163 

AxB 

1.20 

10 

0.12 

0.72 

AxC 

0.37 

4 

0.09 

0.56 

BxC 

0.63 

10 

0.06 

0.38 

AxBxC 

1.62 

20 

0.08 

0.48 

Residual 

26.52 

159 

0.17 

.547 

Total 

Wtt 

?T7 

TABLE  B-9.  ANALYSIS  OF  THE  AVERAGE  VALUE  (A)  AND  VARIANCE  (B)  OF 
THE  POSITION  OF  THE  RUDDER  PEDALS 


(A) 


SOURCE 

SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

_E_ 

n2 

Blocks 

0.255 

3 

0.085 

28.32 

.0000 

.258 

Treatments 

0.260 

53 

0.005 

1.63 

.0112 

.263 

A 

0.015 

2 

0.007 

2.48 

B 

0.010 

5 

0.002 

0.66 

C 

0.116 

2 

0.058 

19.31 

.0000 

.117 

AxB 

0.027 

10 

0.003 

0.91 

AxC 

0.009 

4 

0.002 

0.78 

BxC 

0.041 

10 

0.004 

1.38 

AxBxC 

0.042 

20 

0.002 

0.69 

Residual 

0.476 

159 

0.003 

.481 

Total 

Of 

m 
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TABLE  B-9.  ANALYSIS  OF  THE  AVERAGE  VALUE  (A)  AND  VARIANCE  (B)  OF 
THE  POSITION  OF  THE  RUDDER  PEDALS  (Cont’d) 


(B) 


SOURCE 

SUM  OF  SQUARES 

cif 

MEAN  SQUARE 

F-RATIO 

r\2 

Blocks 

0.0006 

3 

0.0002 

2.00 

Treatments 

0.0066 

53 

0.0001 

1.25 

A 

0.0001 

2 

0.0000 

0.46 

B 

0.0006 

5 

0.0001 

1.79 

C 

0.0016 

2 

0.0008 

11.40 

.0001 

.089 

AxB 

0.0014 

10 

0.0001 

2.05 

.0313 

.07' 

AxC 

0.0000 

4 

0.0000 

0.09 

BxC 

0.0009 

10 

0.0001 

1.29 

AxBxC 

0.0020 

20 

0.0001 

1.48 

Residual 

0.0109 

159 

0.0001 

.606 

Total 

ram 

m 

TABLE  B- 

-10.  ANALYSIS  OF 

THE  AVERAGE  VALUE  (A)  AND  VARIANCE  (B) 

OF 

THE  POSITION  OF  THE  LEFT  THROTTLE 

(A) 

SOURCE 

SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

_E_ 

n2 

Blocks 

295.07 

3 

98.37 

8.19 

.0001 

.091 

Treatments 

1030.41 

53 

19.44 

1.62 

.0118 

.319 

A 

1.85 

2 

0.93 

0.08 

B 

164.57 

5 

32.91 

2.74 

.0208 

.051 

C 

378.84 

2 

189.42 

15.78 

.0000 

.117 

AxB 

93.86 

10 

9.39 

0.78 

AxC 

66.09 

4 

16.52 

1.38 

BxC 

127.26 

10 

12.73 

1.06 

AxBxC 

197.92 

20 

9.90 

0.82 

Residual 

1908.72 

159 

12.00 

.590 

Total 

3734720 

2TT 

(B) 

Blocks 

1823318.00 

3 

607772.66 

22.19 

.0000 

.204 

Treatments 

2770875.00 

53 

52280.66 

1.91 

.0014 

.310 

A 

25772.69 

2 

1 2886. 34 

0.47 

B 

203935.01 

5 

40787.00 

1.49 

C 

1165862.20 

2 

582931.12 

21.29 

.0000 

.130 

AxB 

273127.55 

10 

27312.76 

1.00 

AxC 

66815.78 

4 

16703.95 

0.61 

BxC 

403874.30 

10 

40387.43 

1.47 

AxBxC 

631487.55 

20 

31574.38 

1.15 

Residual 

4354020.20 

159 

27383.78 

.204 

Total 

89'4S5T3V80 

m 
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TABLE  B-ll.  ANALYSIS  OF  THE  AVERAGE  VALUE  (A)  AND  VARIANCE  (B)  OF 
THE  POSITION  OF  THE  RIGHT  THROTTLE 


(A) 


SOURCE 

SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

. _E_ 

n2 

Blocks 

555.32 

3 

185.11 

17.55 

.0000 

.177 

Treatments 

909.42 

53 

17.16 

1.63 

.0110 

.289 

A 

2.13 

2 

1 .06 

0.10 

8 

133.50 

5 

26.70 

2.53 

.0300 

.042 

C 

334.15 

2 

167.07 

15.84 

.0000 

.110 

AxB 

89.67 

10 

8.97 

0.85 

AxC 

58.80 

4 

14.70 

1.39 

BxC 

110.08 

10 

11.01 

1.04 

AxBxC 

181.10 

20 

9.05 

0.86 

Residual 

1676.76 

159 

10.55 

.533 

Total 

TKT.TT 

rnr 

(B) 

Blocks 

1681 344.0 

3 

560448.0 

22.57 

.0000 

.205 

Treatments 

2591764.0 

53 

48901 .2 

1.97 

.0009 

.315 

A 

25939.5 

2 

12969.7 

0.52 

B 

187854.0 

5 

37570.8 

1.51 

C 

1111642.7 

2 

555821  .4 

22.38 

.0000 

.135 

AxB 

251001.7 

10 

25100.2 

1.01 

AxC 

62261  .2 

4 

15565.3 

0.63 

BxC 

360865.3 

10 

36086.5 

1.45 

AxBxC 

592198.7 

20 

29609.9 

1.19 

Residual 

3948198.7 

159 

24831 .4 

.480 

Total 

8221305.2 

m 

TABLE  B-12.  ANALYSIS  OF  THE 

PILOT'S  RATING  OF  HIS  FLIGHT 

SOURCE 

SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

n2 

Blocks 

474.15 

3 

158.05 

38.82  . 

0000 

.296 

Treatments 

479.83 

53 

9.05 

2.22  . 

0002 

.299 

A 

29.36 

2 

14.68 

3.61  . 

0284 

.018 

B 

53.11 

5 

10.61 

2.61  . 

0264 

.033 

C 

207.25 

2 

103.63 

25.45  . 

0000 

.129 

AxB 

59.86 

10 

5.99 

1.47 

AxC 

19.14 

4 

4.78 

1.18 

BxC 

43.81 

10 

4.38 

1.08 

AxBxC 

67.31 

20 

3.37 

0.83 

Residual 

647.35 

159 

4.07 

.404 

Total 

1M3? 

m 
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TABLE  B-l 3.  ANALYSIS  OF  THE  PILOT'S  RATING  OF  THE  CIG  DISPLAY 


SOURCE 

SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

JL_ 

n2 

Blocks 

595.35 

3 

198.45 

103.82 

.0000 

.572 

Treatments 

140.58 

53 

2.65 

1.38 

A 

2.01 

2 

1.00 

0.52 

B 

7.36 

5 

1.47 

0.77 

C 

52.84 

2 

26.42 

13.76 

.0000 

.051 

AxB 

8.05 

10 

0.80 

0.42 

AxC 

9.44 

4 

2.36 

1.23 

BxC 

30.21 

10 

3.02 

1.57 

AxBxC 

30.68 

20 

1.53 

0.80 

Residual 

305.40 

159 

1.92 

.293 

Total 

T04T.73 

2TT 

TABLE  B 

-14.  ANALYSIS  OF  THE  OBSERVER'S  RATINGS  OF  THE 

FLIGHTS 

SOURCE 

SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

P 

n2 

Blocks 

460.80 

3 

153.60 

33.42 

.0000 

.288 

Treatments 

410.33 

53 

7.74 

1.68 

.0075 

.256 

A 

30.33 

2 

15.17 

3.30 

.038 

.019 

B 

19.00 

5 

3.80 

0.83 

C 

176.78 

2 

88.39 

19.23 

.0000 

.110 

AxB 

23.67 

10 

2.37 

0.52 

AxC 

15.72 

4 

3.93 

0.86 

BxC 

38.06 

10 

3.81 

0.83 

AxBxC 

106.78 

20 

5.34 

1.16 

Residual 

730.70 

159 

4.60 

.110 

Total 

16007 

m 

TABLE  B- 

15.  ANALYSIS  OF 

THE  OBSERVER'S  RATINGS  OF  THE 

CIG  DISPLAY 

SOURCE 

SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

_e_ 

n2 

Blocks 

89.65 

3 

29.88 

7.02 

.0004 

.96 

Treatments 

164.83 

53 

3.11 

0.73 

A 

3.25 

2 

1.63 

0.38 

B 

1.50 

5 

0.30 

0.07 

C 

35.11 

2 

17.56 

4.12 

.0000 

.038 

AxB 

15.75 

10 

1.58 

0.37 

AxC 

8.56 

4 

2.14 

0.50 

BxC 

17.72 

10 

1.77 

0.42 

AxBxC 

82.94 

20 

4.15 

0.97 

Residual 

677.35 

159 

4.26 

.727 

Total 

931.63 

m 
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TABLE  B-16. 


ANALYSES  OF  THE  AVERAGE  VALUE,  VARIANCE;  AND  BANDWIDTH 
OF  THE  X-AXIS  DIFFERENCE  OF  POSITION  OF  THE  LEAD  AND 
LAG  AIRCRAFT 


SOURCE  SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

_E_  _aL 

Average  Value 

Between  Groups 
Within  Groups 
Total 

0.243 

0.338 

0.581 

5 

18 

TS 

0.049 

0.019 

2.593 

Variance 

Between  Groups 
Within  Groups 
Total 

0.722 

2.027 

2.749 

5 

18 

2T 

0.144 

0.113 

1.283 

Bandwidth 

Between  Groups 
Within  Groups 
Total 

0.469 

0.436 

0.905 

5 

18 

7J 

0.094 

0.024 

3.869 

.015  .52 

TABLE  B-17. 

ANALYSES  OF 

THE  AVERAGE  VALUE, 

VARIANCE; 

AND  BANDWIDTH 

OF  THE  Y-AXIS  DIFFERENCE  OF  POSITION  OF  THE  LEAD  AND 
LAG  AIRCRAFT 

SOURCE  SUM 

OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

_E_  -111- 

Average  Value 

Between  Groups 
Within  Groups 
Total 

.124 

.312 

.436 

5 

18 

T5 

.025 

.017 

1.431 

Variance 

Between  Groups 
Within  Groups 
Total 

.337 

2.444 

2.761 

5 

18 

2T 

.067 

.137 

0.497 

Bandwidth 

Between  Groups 
Within  Groups 
Total 

.085 

.393 

.478 

5 

18 

T5 

.017 

.022 

0.780 
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TABLE  B— 18.  ANALYSES  OF  THE  AVERAGE  VALUE,  VARIANCE,  AND  BANDWIDTH 
OF  THE  Z-AXIS  DIFFERENCE  OF  POSITION  OF  THE  LEAD  AND  LAG 
AIRCRAFT 


SOURCE  SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

_EL 

n2 

Average  Value 

Between  Groups 
Within  Groups 
Total 

0.729 

0.793 

1.522 

5 

18 

27 

0.146 

0.044 

3.310 

.027 

.48 

Variance 

Between  Groups 
Within  Groups 
Total 

1.236 

1.502 

2.738 

5 

18 

27 

0.247 

0.083 

2.961 

.039 

.45 

Bandwl dth 

Between  Groups 
Within  Groups 
Total 

0.186 

0.393 

0.579 

5 

18 

27 

0.037 

0.022 

1.700 

TABLE  B-19. 

ANALYSES  OF  THE  AVERAGE  VALUE,  VARIANCE,  AND  BANDWIDTH 
OF  THE  DIFFERENCE  OF  ROLL  ANGLE  OF  THE  LEAD  AND  LAG 
AIRCRAFT 

SOURCE  SUM 

OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

_E_ 

n2 

Average  Value 

Between  Groups 
Within  Groups 
Total 

0.753 

1.285 

2V0S8 

5 

18 

27 

0.151 

0.071 

2.110 

Variance 

Between  Groups 
Within  Groups 
Total 

3.145 

0.811 

05F 

5 

18 

27 

0.629 

0.045 

13.961 

.000 

.79 

Bandwidth 

Between  Groups 
Within  Groups 
Total 

0.958 

0. 309 

1. ^67 

5 

18 

27 

0.192 

0.017 

11.171 

.000 

.76 
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TABLE  B-20.  ANALYSES  OF  THE  AVERAGE  VALUE,  VARIANCE,  AND  BANDWIDTH 
OF  THE  DIFFERENCE  OF  PITCH  ANGLE  OF  THE  LEAD  AND  LAG 
AIRCRAFT 


SOURCE  SUM 

OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

P n2 

Average  Value 

Between  Groups 

0.530 

5 

0.106 

2.663 

Within  Groups 

0.716 

18 

0.040 

Total 

w 

37 

Variance 

Between  Groups 

3.783 

5 

0.757 

66.842 

.000  .95 

Within  Groups 

0.204 

18 

0.011 

Total 

■037 

37 

Bandwi dth 

Between  Groups 

0.628 

5 

0.126 

37.550 

.000  .91 

Within  Groups 

0.060 

18 

0.003 

Total 

.688 

37 

TABLE  B— 21 . 

ANALYSES  OF  THE  AVERAGE  VALUE, 

VARIANCE, 

AND  BANDWIDTH 

OF  THE  DIFFERENCE 

OF  YAW  ANGLE  OF  THE  LEAD  AND  LAG 

AIRCRAFT 

SOURCE  SUM 

OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

P n2 

Average  Value 

Between  Groups 

0.230 

5 

0.046 

2.072 

Within  Groups 

0.398 

18 

0.022 

Total 

0.628 

37 

Variance 

Between  Groups 

1.585 

5 

0.317 

5.190 

.004  .59 

Within  Groups 

1.100 

18 

0.061 

Total 

2.685 

37 

Bandwidth 

Between  Groups 

0.233 

5 

0.047 

1.833 

Within  Groups 

0.456 

18 

0.025 

Total 

0.589 

37 

( 
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TABLE  B-22.  ANALYSES  OF  THE  AVERAGE  VALUE,  VARIANCE,  AND  BANDWIDTH 
OF  THE  LATERAL  MOVEMENTS  OF  THE  CONTROL  STICK 


SOURCE  SUM 

OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

_EL 

n2 

Average  Value 

Between  Groups 
Within  Groups 
Total 

0.4$1 

0.287 

0.758 

5 

18 

IT 

0.096 

0.016 

6.039 

.002 

.63 

Vari ance 

Between  Groups 
Within  Groups 
Total 

9.316 

0.438 

9775T 

5 

18 

IT 

1.863 

0.024 

76.475 

.000 

.96 

Bandwidth 

Between  Groups 
Within  Groups 
Total 

0.148 

0.122 

0.270 

5 

18 

IT 

0.030 

0.007 

4.337 

.009 

.55 

TABLE  B-23. 

ANALYSES  OF  THE  AVERAGE  VALUE,  VARIANCE,  AND  BANDWIDTH 
OF  THE  LONGITUDINAL  MOVEMENTS  OF  THE  CONTROL  STICK 

SOURCE  SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

_e_ 

n2 

Average  Value 

Between  Groups 
Within  Groups 
Total 

0.000 

0.002 

O.fitifc 

5 

18 

IT 

0.000 

0.000 

0.226 

Variance 

Between  Grouas 
Within  Groups 
Total 

3.942 

0.570 

4.612 

5 

18 

IT 

0.788 

0.032 

24.894 

.000 

.87 

Bandwidth 

Between  Groups 
Within  Groups 
Total 

0.517 

0.530 

77W7 

5 

18 

IT 

0.103 

0.029 

3.516 

.021 

.49 
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TABLE  B-24.  ANALYSES  OF  THE  AVERAGE  VALUE,  VARIANCE,  AND  BANDWIDTH  OF 
THE  POSITION  OF  THE  RUDDER  PEDALS 


SOURCE  SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

-EL  _£i_ 

Average  Value 

Between  Groups 

0.234 

5 

0.047 

0.660 

Within  Groups 

1.275 

18 

0.071 

Total 

7309- 

23" 

Variance 

Between  Groups 

1.365 

5 

0.273 

6.420 

.002  .64 

Within  Groups 

0.765 

18 

0.043 

Total 

2.130 

2T 

Bandwidth 

Between  Groups 

0.095 

5 

0.019 

0.232 

Within  Groups 

1.479 

18 

0.082 

Total 

1 .574 

23" 

TABLE  B-25. 

ANALYSES  OF  THE 

AVERAGE  VALUE, 

VARIANCE, 

AND  BANDWIDTH 

OF  THE  POSITION 

OF  THE  LEFT  THROTTLE 

SOURCE  SUM 

OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

P n2 

Average  Value 

Between  Groups 

0.003 

5 

0.0006 

3.530 

.021  .60 

Within  Groups 

0.002 

18 

0.0001 

Total 

0. 005 

2T 

Variance 

Between  Groups 

1.145 

5 

0.229 

3.408 

.024  .49 

Within  Groups 

1.209 

18 

0.067 

Total 

osr 

23" 

Bandwidth 

Between  Groups 

0.540 

5 

0.108 

1.641 

Within  Groups 

1.185 

18 

0.066 

Total 

77725- 

2T 

( 
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TABLE  B-26.  ANALYSES  OF  THE  AVERAGE  VALUE,  VARIANCE,  AND  BANDWIDTH 
OF  THE  POSITION  OF  THE  RIGHT  THROTTLE 


SOURCE  SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

_E n i_ 

Averaqe  Value 

Between  Groups 
Within  Groups 
Total 

0.002 

0.003 

0.0O5 

5 

18 

27 

0.000 

0.000 

3.035 

.037  .40 

Variance 

Between  Groups 
Within  Groups 
Total 

1.184 

1.220 

2.404 

5 

18 

27 

0.237 

0.067 

3.493 

.022  .49 

Bandwi dth 

Between  Groups 
Within  Groups 
Total 

0.398 

1.447 

1.845 

5 

18 

27 

0.080 

0.080 

0.991 

TABLE  B-27. 

ANALYSES  OF  THE 
OF  THE  FLIGHTS 

PILOTS'  RATINGS  OF  THE  CIG 

DISPLAY  AND 

SOURCE  SUM  OF  SQUARES 

df 

MEAN  SQUARE 

F-RATIO 

_E ni_ 

CIG  Display 

Between  Groups 
Within  Groups 
Total 

15.64 

1.34 

15755" 

5 

18 

21 

3.128 

0.074 

42.02 

.000  .92 

Fliqhts 

3e tween  Groups 
Within  Groups 
Total 

5.05 

2.16 

T72T 

5 

18 

2T 

1.010 

0.120 

8.41 

.000  .70 
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